-- Logs begin at Mon 2022-08-08 07:52:42 UTC, end at Mon 2022-08-08 09:29:54 UTC. -- Aug 08 08:22:17.992144 dv-cinder-836973-17-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 08 08:22:18.002353 dv-cinder-836973-17-iscsi devstack@keystone.service[80248]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 08 08:22:18.004550 dv-cinder-836973-17-iscsi devstack@keystone.service[80248]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 8 08:22:18 2022] *** Aug 08 08:22:18.004550 dv-cinder-836973-17-iscsi devstack@keystone.service[80248]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 08 08:22:18.004550 dv-cinder-836973-17-iscsi devstack@keystone.service[80248]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 08 08:22:18.004550 dv-cinder-836973-17-iscsi devstack@keystone.service[80248]: nodename: dv-cinder-836973-17-iscsi Aug 08 08:22:18.004550 dv-cinder-836973-17-iscsi devstack@keystone.service[80248]: machine: x86_64 Aug 08 08:22:18.004550 dv-cinder-836973-17-iscsi devstack@keystone.service[80248]: clock source: unix Aug 08 08:22:18.004550 dv-cinder-836973-17-iscsi devstack@keystone.service[80248]: pcre jit disabled Aug 08 08:22:18.004550 dv-cinder-836973-17-iscsi devstack@keystone.service[80248]: detected number of CPU cores: 4 Aug 08 08:22:18.004550 dv-cinder-836973-17-iscsi devstack@keystone.service[80248]: current working directory: / Aug 08 08:22:18.004550 dv-cinder-836973-17-iscsi devstack@keystone.service[80248]: detected binary path: /usr/bin/uwsgi-core Aug 08 08:22:18.004550 dv-cinder-836973-17-iscsi devstack@keystone.service[80248]: your processes number limit is 47680 Aug 08 08:22:18.004550 dv-cinder-836973-17-iscsi devstack@keystone.service[80248]: your memory page size is 4096 bytes Aug 08 08:22:18.005227 dv-cinder-836973-17-iscsi devstack@keystone.service[80248]: detected max file descriptor number: 2048 Aug 08 08:22:18.005227 dv-cinder-836973-17-iscsi devstack@keystone.service[80248]: lock engine: pthread robust mutexes Aug 08 08:22:18.005227 dv-cinder-836973-17-iscsi devstack@keystone.service[80248]: thunder lock: enabled Aug 08 08:22:18.005227 dv-cinder-836973-17-iscsi devstack@keystone.service[80248]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 08 08:22:18.005227 dv-cinder-836973-17-iscsi devstack@keystone.service[80248]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 08 08:22:18.045950 dv-cinder-836973-17-iscsi devstack@keystone.service[80248]: Python main interpreter initialized at 0x5610c131e9c0 Aug 08 08:22:18.045950 dv-cinder-836973-17-iscsi devstack@keystone.service[80248]: python threads support enabled Aug 08 08:22:18.045950 dv-cinder-836973-17-iscsi devstack@keystone.service[80248]: your server socket listen backlog is limited to 100 connections Aug 08 08:22:18.045950 dv-cinder-836973-17-iscsi devstack@keystone.service[80248]: your mercy for graceful operations on workers is 90 seconds Aug 08 08:22:18.046338 dv-cinder-836973-17-iscsi devstack@keystone.service[80248]: mapped 403077 bytes (393 KB) for 2 cores Aug 08 08:22:18.046438 dv-cinder-836973-17-iscsi devstack@keystone.service[80248]: *** Operational MODE: preforking *** Aug 08 08:22:18.047421 dv-cinder-836973-17-iscsi devstack@keystone.service[80248]: *** uWSGI is running in multiple interpreter mode *** Aug 08 08:22:18.047421 dv-cinder-836973-17-iscsi devstack@keystone.service[80248]: spawned uWSGI master process (pid: 80248) Aug 08 08:22:18.047626 dv-cinder-836973-17-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 08 08:22:18.051098 dv-cinder-836973-17-iscsi devstack@keystone.service[80248]: spawned uWSGI worker 1 (pid: 80261, cores: 1) Aug 08 08:22:18.051098 dv-cinder-836973-17-iscsi devstack@keystone.service[80248]: spawned uWSGI worker 2 (pid: 80262, cores: 1) Aug 08 08:22:18.051098 dv-cinder-836973-17-iscsi devstack@keystone.service[80248]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 08 08:22:23.595694 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 08:22:23.595694 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: from cryptography.utils import int_from_bytes Aug 08 08:22:23.596484 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 08:22:23.596484 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: from cryptography.utils import int_from_bytes Aug 08 08:22:23.596588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 08:22:23.596588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: from cryptography.utils import int_from_bytes Aug 08 08:22:23.596588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 08:22:23.596588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: from cryptography.utils import int_from_bytes Aug 08 08:22:23.645870 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 08 08:22:23.646056 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 08 08:22:23.646149 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 08 08:22:23.646243 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 08 08:22:23.646321 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] command line args: [] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 08 08:22:23.646403 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] command line args: [] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 08 08:22:23.646540 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 08 08:22:23.646660 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 08 08:22:23.646748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ================================================================================ {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 08 08:22:23.646828 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ================================================================================ {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 08 08:22:23.647031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] admin_token = **** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.647123 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] admin_token = **** {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.647308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] config_dir = [] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.647387 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] config_dir = [] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.647495 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.647581 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.647678 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] config_source = [] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.647764 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] config_source = [] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.647858 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] debug = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.647942 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] debug = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.648055 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.648384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.648515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.648624 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.648730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.648835 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.648943 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.649059 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.649183 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.649288 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] list_limit = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.649384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.649499 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] log_config_append = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.649629 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.649743 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] list_limit = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.649849 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] log_dir = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.649948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] log_config_append = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.650062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] log_file = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.650160 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.650257 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.650364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] log_dir = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.650450 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] log_file = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.650575 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.650658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.650735 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.650885 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.650981 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.651060 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.651224 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.651303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.651467 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.651597 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.651717 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.651833 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.651950 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.652072 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.652188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.652310 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.652418 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.652531 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.652652 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.652765 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.652885 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.653009 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.653117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.653274 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.653399 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.653524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.653789 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.653911 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.654032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] publish_errors = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.654140 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.654264 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.654388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.654500 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.654621 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.654795 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.655027 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.655208 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.655208 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] publish_errors = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.655438 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] standard_threads = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.655557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.655697 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.655827 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.655956 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.656072 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.656186 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.656314 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.656487 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] use_journal = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.656612 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] use_json = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.656728 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.656844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] use_stderr = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.656959 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] standard_threads = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.657084 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] use_syslog = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.657084 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.657347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.657463 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.657586 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.657709 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.657837 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.657837 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] use_journal = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.658068 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.658185 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] use_json = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.658300 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.658423 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] use_stderr = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.658620 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.658739 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] use_syslog = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.658855 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.658972 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 08:22:23.659094 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.659210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.659337 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.659337 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.659557 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.659679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.659796 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.659911 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.660038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.660155 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.660155 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.660378 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.660503 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.660620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.660747 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.660865 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.660986 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.661103 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.661331 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.661464 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.661589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.661720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.661837 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.661952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.662073 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.662198 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.662322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.662512 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.662675 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.662801 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.662917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.663044 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.663216 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.663335 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.663456 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.663574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.663690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.663814 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.663931 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.664121 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.664241 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.664395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.664514 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.664706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.664816 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.664938 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.665061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.665266 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.665388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.665517 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.665676 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.665810 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.665810 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.666037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.666153 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.666276 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.666400 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.666516 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.666631 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.666747 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.666861 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.666988 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.667111 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.667240 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.667363 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.667479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.667610 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.667725 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.667858 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.667970 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.668108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.668220 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.668324 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.668454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.668560 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.668745 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.668833 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.668995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.669087 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.669249 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.669332 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.669481 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.669649 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.669780 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.669931 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.670061 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.670170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.670285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.670401 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.670540 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.670674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.670804 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.670924 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.671050 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.671168 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.671287 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.671404 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.672042 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.672253 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.672412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.672533 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.672684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.672813 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.672938 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.673067 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.673206 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.673327 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.673444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.673562 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.673695 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.673820 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.673938 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.674056 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.674180 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.674298 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.674424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.674542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.674659 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.674789 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.674897 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.675013 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.675137 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.675264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.675391 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.675510 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.675650 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.675777 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.675921 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.676051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.676170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.676287 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.676404 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.676521 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.676637 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.676830 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.676970 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.677093 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.677242 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.677360 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.677477 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.677677 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.677798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.677915 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.678032 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.678147 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.678271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.678400 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.678525 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.678643 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.678761 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.678879 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.678995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.679125 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.679257 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.679383 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.679508 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.679632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.679750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.679867 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.679992 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.680110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.680234 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.680352 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.680469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.680586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.680714 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.680833 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.680950 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.681115 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.681257 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.681374 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.681504 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.681630 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.681749 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.681866 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.682027 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.682148 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.682284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.682403 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.682521 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.682641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.682758 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.682875 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.683000 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.683162 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.683282 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.683424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.683546 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.683669 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.683796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.683915 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.684033 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.684156 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] auth.external = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.684275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.684392 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.684557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.684680 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.684845 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.684962 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.685078 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] auth.password = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.685246 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.685379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] auth.token = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.685504 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.685623 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.685763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.685883 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.686001 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.686156 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.686284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.686460 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.686581 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.686707 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.686857 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.686978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.687206 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] credential.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.687850 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.687979 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.688107 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] auth.external = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.688278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.688400 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.688519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.688644 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.688767 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.688884 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.689011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.689131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] auth.password = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.689273 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.689405 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] auth.token = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.689526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.689644 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.689761 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.689880 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.690005 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.690132 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.690251 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.690369 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.690487 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.690605 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.690732 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.690852 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.690970 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.691089 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.691207 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.691325 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] credential.caching = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.691450 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.691570 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.691687 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] federation.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.691804 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.691923 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.692040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.692169 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.692297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.692415 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.692533 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.692651 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.692768 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.692963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.693086 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.693234 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.693353 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.693470 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.693606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.693735 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.693854 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.693972 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.694090 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.694208 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.694325 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.694450 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.694569 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.694687 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.694804 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.694922 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.695031 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.695156 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.695282 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] federation.caching = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.695408 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.695527 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.695644 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.695769 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.695899 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.696019 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.696138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.696261 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.696380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.696497 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.696624 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.696744 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.696862 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.696979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.697097 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.697267 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.697399 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.697399 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.697623 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.697741 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.697859 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.697977 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.698104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.698225 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.698343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity.caching = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.698460 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.698578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.698695 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.698822 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.698822 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.699046 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.699171 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.699297 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.699416 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.699542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.699661 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.699778 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.699897 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.700015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.700015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.700246 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.700365 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.700473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.700595 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.700714 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.700844 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.700963 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.700963 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.701198 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.701198 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.701421 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.701552 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.701672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.701790 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.701898 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.702014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.702131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.702258 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.702378 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.702496 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.702642 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.702791 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.702916 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.703043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.703162 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.703281 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.703398 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.703517 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.703635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.703635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.704124 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.704244 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.704361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.704480 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.704855 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.704989 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.705115 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.705251 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.705370 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.705488 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.705607 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.705738 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.705865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.705983 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.706101 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.706221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.706345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.706472 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.706593 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.706710 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.706828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.706946 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.707063 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.707192 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.707311 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.707429 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.707547 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.707666 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.707783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.707909 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.708029 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.708147 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.708255 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.708371 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.708489 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.708615 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.708735 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.708853 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.708970 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.709087 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.709220 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.709355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.709481 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.709600 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.709719 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.709837 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.709954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.710080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.710199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.710317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.710434 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.710552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.710670 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.710797 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.710916 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.711033 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.711151 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.711268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.711386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.711513 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.711632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.711764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.711882 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.712000 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.712118 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.712245 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.712364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.712482 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.712599 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.712717 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.712834 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.712961 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.713080 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.713222 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.713342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.713460 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.713578 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.713709 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.713828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.713947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.714065 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.714183 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.714308 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.714427 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.714545 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.714669 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.714789 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.714907 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.715034 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.715154 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.715271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.715389 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.715506 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.715624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.715750 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.715870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.715988 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.716107 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.716228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.716345 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.716542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] resource.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.716670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.716790 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.716907 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.717025 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.717161 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.717293 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.717414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.717537 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.717656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.717783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.717902 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.718028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.718148 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.718273 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.718416 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.718535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.718653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.718780 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.718900 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.719017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] role.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.719134 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.719252 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] role.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.719369 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.719495 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.719614 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.719732 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.719850 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.719968 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.720086 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] resource.caching = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.720213 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.720333 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.720450 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.720450 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.720672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.720789 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.720919 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.721038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.721205 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.721327 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.721444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.721562 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.721692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.721813 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.721930 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.722047 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.722165 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.722282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.722408 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.722527 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.722645 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] role.caching = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.722645 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] role.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.722864 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.722981 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.723107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.723226 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.723345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.723463 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.723581 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.723698 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.723698 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.723928 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.724046 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.724164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.724281 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.724408 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.724527 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.724644 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.724761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.724879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.724996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.725121 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.725255 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.725372 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.725491 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.725608 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.725725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.725855 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.725974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.726092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.726092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.726316 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.726433 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.726560 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.726685 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.726804 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.726922 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.727038 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.727156 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.727156 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] token.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.727386 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.727503 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.727621 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.727739 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.727856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.727982 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.728101 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.728218 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.728335 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.728453 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.728570 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.728697 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.728816 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.728934 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.729050 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.729237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.729358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.729489 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.729608 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.729727 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.729844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.729961 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.730078 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.730204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.730323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.730440 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.730557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.730675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.730791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.730917 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.731036 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] token.caching = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.731153 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.731270 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.731388 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.731505 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.731630 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.connection = **** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.731759 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.731887 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.732007 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.732124 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.732242 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.732359 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.732484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.732603 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.732720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.732838 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.732956 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.733074 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.733215 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.733336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.733454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.733571 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.733689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.733807 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.733933 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.734615 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.734737 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.734855 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.734974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.735092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.735220 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.735341 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.735459 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.735577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.735694 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.735813 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.735941 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.connection = **** {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.736069 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.736266 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.736387 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.736506 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.736624 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.736743 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.736869 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80261) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 08 08:22:23.736990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.736990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.736990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.736990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.736990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.736990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.736990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.736990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.736990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.737979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.737979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.737979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.737979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.737979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.737979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 08:22:23.737979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80262) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 08 08:22:23.763928 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.764155 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.764383 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.767634 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.767865 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.768092 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.774144 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.777825 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.802892 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.803117 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.804598 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.804831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.855415 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.855946 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.906284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.906589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.907042 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.907278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.973809 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.973959 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.974084 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.974172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.974346 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.974462 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.974610 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.974689 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.974762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.974845 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.974981 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.975196 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.997763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.997972 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.998096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.998181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.998327 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.998408 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.998551 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.998631 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.998771 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.998852 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.998988 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.999066 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.999207 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:23.999420 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:24.010679 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:24.011003 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:24.023914 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:24.024140 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:24.024357 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:24.024445 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:24.024575 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:24.024661 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:24.024790 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:24.024874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:24.025002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:24.025087 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:24.025237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80261) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:24.025322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:24.025478 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:24.025694 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80262) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 08:22:24.041907 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.042113 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 08:22:24.042454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.042652 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 08:22:24.042857 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.042942 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.043068 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 08:22:24.043171 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.043410 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.043491 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.043666 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.043750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 08:22:24.043990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.044254 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.044512 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.044769 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.055148 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.055423 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.055689 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.055968 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.056410 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.056690 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.056960 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.057267 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.063463 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.064981 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.066543 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.066826 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.067105 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.067387 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.067671 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.067928 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.068071 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.068194 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.068356 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.068461 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.068640 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.068731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.068922 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.069004 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.069226 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.069312 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.069487 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.069756 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.070026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.070294 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.070571 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.070835 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.092290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.094306 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.094587 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.094988 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.096472 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.096758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.099159 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.099438 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.099715 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.101390 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.101672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.101953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.106320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.106605 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.108532 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.108881 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.111511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.111804 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.112084 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.112375 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.112658 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.112943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.113198 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.113282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.113494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.113574 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.113789 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.114084 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.114358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.114644 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.114919 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.115197 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.129803 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.130075 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.131794 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.132071 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.134117 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.136024 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.137235 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.137530 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.139193 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.139490 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.141976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.144124 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.145121 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.147311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.148269 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.148564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.148853 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.149161 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.149445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.149728 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.150486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.150801 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.151089 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.151386 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.151674 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.151953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.166233 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.166517 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.166781 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.167040 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.168525 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.168814 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.169080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.169367 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.174010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.175558 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.175832 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.176516 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.178113 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.178393 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.178653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.178936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.179228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.179523 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.179796 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.181221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.181507 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.181799 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.182096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.182372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.191690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.191969 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.192242 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.192518 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.192793 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.193066 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.193362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.194307 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.194592 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.194871 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.195152 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.195433 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.195712 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.195987 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.210334 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.212813 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.213303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.215796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.216221 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.217813 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.219309 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.219973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.220264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.220563 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.220872 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.222405 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.222696 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.222987 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.226996 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.228550 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.229408 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.231010 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.231494 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.231780 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.232061 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.232332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.234067 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.234359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.234646 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.234918 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.240544 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.240921 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.241386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.243023 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.243358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.243639 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.248199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80261) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.248481 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.248762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.249034 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.249330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.250150 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.250387 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.250648 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.251029 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.251205 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80262) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 08:22:24.251307 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.251489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.251577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.251767 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.251849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.252048 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.252127 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.252318 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.252406 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80261) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.252588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.252860 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.253126 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.253528 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.253831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.254093 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.254357 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.254632 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.254892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80262) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 08:22:24.285794 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5610c131e9c0 pid: 80261 (default app) Aug 08 08:22:24.288462 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5610c131e9c0 pid: 80262 (default app) Aug 08 08:22:24.289015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb1a07-e281-4ed4-8e5e-3019cee6308b None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:24.289320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb1a07-e281-4ed4-8e5e-3019cee6308b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:24.289591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb1a07-e281-4ed4-8e5e-3019cee6308b None None] PATH_INFO: `/v3` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:24.293891 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1/1] 10.222.0.33 () {50 vars in 791 bytes} [Mon Aug 8 08:22:24 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:32.808378 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ac023-7fd9-4f29-8696-4e1766e6f35b None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:32.808874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ac023-7fd9-4f29-8696-4e1766e6f35b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:32.808874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ac023-7fd9-4f29-8696-4e1766e6f35b None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:32.813883 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1/2] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:22:32 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:22:32.820361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504dee1-14ee-40b9-a530-365dcc0311be None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:32.820788 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504dee1-14ee-40b9-a530-365dcc0311be None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:32.820907 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504dee1-14ee-40b9-a530-365dcc0311be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:32.902159 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: INFO dbcounter [None req-5504dee1-14ee-40b9-a530-365dcc0311be None None] Registered counter for database keystone Aug 08 08:22:32.926733 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG oslo_db.sqlalchemy.engines [None req-5504dee1-14ee-40b9-a530-365dcc0311be None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80261) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 08:22:32.932969 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writer thread running {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 08:22:33.114181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG passlib.handlers.bcrypt [None req-5504dee1-14ee-40b9-a530-365dcc0311be None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80261) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 08 08:22:33.114530 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG passlib.handlers.bcrypt [None req-5504dee1-14ee-40b9-a530-365dcc0311be None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80261) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 08 08:22:33.168656 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-5504dee1-14ee-40b9-a530-365dcc0311be None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:22:33.170298 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5504dee1-14ee-40b9-a530-365dcc0311be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:22:33.300769 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 2/3] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:22:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 08 08:22:33.411787 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ac023-7fd9-4f29-8696-4e1766e6f35b None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:33.481496 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: INFO dbcounter [None req-884ac023-7fd9-4f29-8696-4e1766e6f35b None None] Registered counter for database keystone Aug 08 08:22:33.503349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG oslo_db.sqlalchemy.engines [None req-884ac023-7fd9-4f29-8696-4e1766e6f35b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80262) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 08:22:33.509346 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writer thread running {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 08:22:33.626004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ac023-7fd9-4f29-8696-4e1766e6f35b None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:33.630158 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ddeb7-365e-42c3-a301-3666f8a25916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:33.630771 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ddeb7-365e-42c3-a301-3666f8a25916 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:33.631043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ddeb7-365e-42c3-a301-3666f8a25916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:33.631310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ddeb7-365e-42c3-a301-3666f8a25916 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:33.639198 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ddeb7-365e-42c3-a301-3666f8a25916 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:33.690709 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d87ddeb7-365e-42c3-a301-3666f8a25916 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 08 08:22:33.690983 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ddeb7-365e-42c3-a301-3666f8a25916 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:33.700327 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 2/4] 10.222.0.33 () {58 vars in 1324 bytes} [Mon Aug 8 08:22:33 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 08:22:33.706844 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504dee1-14ee-40b9-a530-365dcc0311be None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:33.721352 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504dee1-14ee-40b9-a530-365dcc0311be None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:33.725762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d21ca-735f-4d25-bc29-60b8e59eec1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:33.726876 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d21ca-735f-4d25-bc29-60b8e59eec1a admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:33.727161 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d21ca-735f-4d25-bc29-60b8e59eec1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:33.727435 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d21ca-735f-4d25-bc29-60b8e59eec1a admin admin] PATH_INFO: `/v3/projects/785791bda4044663bfe6cd25439a8358` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:33.728857 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065d21ca-735f-4d25-bc29-60b8e59eec1a admin admin] RBAC: Authorizing `identity:get_project(project_id=785791bda4044663bfe6cd25439a8358)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:33.778822 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: WARNING keystone.common.rbac_enforcer.enforcer [None req-065d21ca-735f-4d25-bc29-60b8e59eec1a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 08 08:22:33.779167 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065d21ca-735f-4d25-bc29-60b8e59eec1a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:33.784296 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 3/5] 10.222.0.33 () {56 vars in 1207 bytes} [Mon Aug 8 08:22:33 2022] GET /identity/v3/projects/785791bda4044663bfe6cd25439a8358 => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:35.744918 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e627a-4bc8-494d-8493-d0f9be74438a None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:35.745425 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e627a-4bc8-494d-8493-d0f9be74438a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:35.745425 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e627a-4bc8-494d-8493-d0f9be74438a None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:35.750097 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 3/6] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:22:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:22:35.755638 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0434ef0-4c1f-4b30-98ed-919f75a8ac25 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:35.756037 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0434ef0-4c1f-4b30-98ed-919f75a8ac25 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:35.756177 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0434ef0-4c1f-4b30-98ed-919f75a8ac25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:35.777060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d0434ef0-4c1f-4b30-98ed-919f75a8ac25 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:22:35.778593 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d0434ef0-4c1f-4b30-98ed-919f75a8ac25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:22:35.786481 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 4/7] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:22:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 08 08:22:35.807928 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e627a-4bc8-494d-8493-d0f9be74438a None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:35.820553 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e627a-4bc8-494d-8493-d0f9be74438a None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:35.824645 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77457aa3-51dd-4fce-b63e-fb6ed32f5473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:35.825265 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77457aa3-51dd-4fce-b63e-fb6ed32f5473 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:35.825544 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77457aa3-51dd-4fce-b63e-fb6ed32f5473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:35.825816 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77457aa3-51dd-4fce-b63e-fb6ed32f5473 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:35.833935 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77457aa3-51dd-4fce-b63e-fb6ed32f5473 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:35.835684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77457aa3-51dd-4fce-b63e-fb6ed32f5473 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:35.844598 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 4/8] 10.222.0.33 () {58 vars in 1324 bytes} [Mon Aug 8 08:22:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 08:22:35.849839 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0434ef0-4c1f-4b30-98ed-919f75a8ac25 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:35.859144 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0434ef0-4c1f-4b30-98ed-919f75a8ac25 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:35.863471 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89066851-49a5-4afe-bd8f-f8d61bbf8745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:35.864085 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89066851-49a5-4afe-bd8f-f8d61bbf8745 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:35.864368 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89066851-49a5-4afe-bd8f-f8d61bbf8745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:35.864642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89066851-49a5-4afe-bd8f-f8d61bbf8745 admin admin] PATH_INFO: `/v3/users/63f02a13b768446ca706d7594e04d3fc` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:35.865963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89066851-49a5-4afe-bd8f-f8d61bbf8745 admin admin] RBAC: Authorizing `identity:get_user(user_id=63f02a13b768446ca706d7594e04d3fc)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:35.867779 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89066851-49a5-4afe-bd8f-f8d61bbf8745 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:35.869332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 5/9] 10.222.0.33 () {56 vars in 1198 bytes} [Mon Aug 8 08:22:35 2022] GET /identity/v3/users/63f02a13b768446ca706d7594e04d3fc => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:37.900944 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5402056-b3bb-4a2a-8380-601895272a5a None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:37.901530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5402056-b3bb-4a2a-8380-601895272a5a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:37.901530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5402056-b3bb-4a2a-8380-601895272a5a None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:37.902988 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 5/10] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:22:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:22:37.908124 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b968648-830d-410f-aa82-c4e8087c028b None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:37.908533 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b968648-830d-410f-aa82-c4e8087c028b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:37.908824 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b968648-830d-410f-aa82-c4e8087c028b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:37.927053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-1b968648-830d-410f-aa82-c4e8087c028b None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:22:37.928229 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1b968648-830d-410f-aa82-c4e8087c028b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:22:37.948668 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab94fc-eaed-4035-bdd3-6b13c3e85a38 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:37.948954 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab94fc-eaed-4035-bdd3-6b13c3e85a38 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:37.949266 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab94fc-eaed-4035-bdd3-6b13c3e85a38 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:37.950585 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 6/11] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:22:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 08:22:37.956097 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bba48-98fc-4be3-90fb-246697428eee None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:37.956387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bba48-98fc-4be3-90fb-246697428eee None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:37.956664 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bba48-98fc-4be3-90fb-246697428eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:37.999203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG passlib.handlers.bcrypt [None req-528bba48-98fc-4be3-90fb-246697428eee None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80262) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 08 08:22:38.000344 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG passlib.handlers.bcrypt [None req-528bba48-98fc-4be3-90fb-246697428eee None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80262) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 08 08:22:38.008458 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 6/12] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:22:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:22:38.030617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b968648-830d-410f-aa82-c4e8087c028b None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:38.045304 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-528bba48-98fc-4be3-90fb-246697428eee None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:22:38.047363 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-528bba48-98fc-4be3-90fb-246697428eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:22:38.053247 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:38.132361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b968648-830d-410f-aa82-c4e8087c028b None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:38.146305 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 7/13] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:22:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:22:38.167830 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528bba48-98fc-4be3-90fb-246697428eee None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:38.167980 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e306aa0a-ca01-4de5-88bb-3ae61ba04bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:38.168597 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306aa0a-ca01-4de5-88bb-3ae61ba04bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:38.168876 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306aa0a-ca01-4de5-88bb-3ae61ba04bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:38.169171 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306aa0a-ca01-4de5-88bb-3ae61ba04bc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:38.181115 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e306aa0a-ca01-4de5-88bb-3ae61ba04bc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:22:38.190275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-e306aa0a-ca01-4de5-88bb-3ae61ba04bc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:22:38.190275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:22:38.190275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:22:38.190275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:22:38.190275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:22:38.190275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:22:38.190275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:22:38.190275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:22:38.190275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:22:38.190275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:22:38.190275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:22:38.190275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:22:38.190275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:22:38.190275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:22:38.190275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:22:38.190275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:22:38.191301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:22:38.191301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:22:38.191301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:22:38.191301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:22:38.191301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:22:38.191301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:22:38.191301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:22:38.191301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:22:38.191301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:22:38.191301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:22:38.191301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:22:38.191301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:22:38.191301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:22:38.191301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:22:38.191301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:22:38.191301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:22:38.192128 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:22:38.192128 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:22:38.192128 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:22:38.192128 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:22:38.192128 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:22:38.192128 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:22:38.192128 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:22:38.192128 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:22:38.192128 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:22:38.192128 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:22:38.192128 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:22:38.192128 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:22:38.196324 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 7/14] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:22:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:22:38.197120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e306aa0a-ca01-4de5-88bb-3ae61ba04bc7 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:38.267387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528bba48-98fc-4be3-90fb-246697428eee None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:38.269664 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:38.278537 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:38.296982 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e306aa0a-ca01-4de5-88bb-3ae61ba04bc7 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:38.304098 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e93ef-99d7-4bba-806e-88f26d73c217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:38.304957 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e93ef-99d7-4bba-806e-88f26d73c217 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:38.305265 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e93ef-99d7-4bba-806e-88f26d73c217 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:38.305541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e93ef-99d7-4bba-806e-88f26d73c217 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:38.332316 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419ef00-fae5-4496-920f-935403a609f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:38.332916 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419ef00-fae5-4496-920f-935403a609f6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:38.333220 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419ef00-fae5-4496-920f-935403a609f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:38.333489 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419ef00-fae5-4496-920f-935403a609f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:38.334157 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a419ef00-fae5-4496-920f-935403a609f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:22:38.334493 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a419ef00-fae5-4496-920f-935403a609f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:38.335877 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a419ef00-fae5-4496-920f-935403a609f6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:38.347836 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 8/15] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:22:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:22:38.353464 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419ef00-fae5-4496-920f-935403a609f6 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:38.373164 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763e93ef-99d7-4bba-806e-88f26d73c217 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:38.375208 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763e93ef-99d7-4bba-806e-88f26d73c217 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:38.453241 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419ef00-fae5-4496-920f-935403a609f6 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:38.476434 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 8/16] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:22:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 08:22:38.482416 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e93ef-99d7-4bba-806e-88f26d73c217 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:38.486546 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:38.588416 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e93ef-99d7-4bba-806e-88f26d73c217 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:38.594894 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929ebe1-bb37-47cb-8ff6-4382685270f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:38.595520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929ebe1-bb37-47cb-8ff6-4382685270f3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:38.595797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929ebe1-bb37-47cb-8ff6-4382685270f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:38.596068 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929ebe1-bb37-47cb-8ff6-4382685270f3 None admin] PATH_INFO: `/v3/users/63f02a13b768446ca706d7594e04d3fc` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:38.597227 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929ebe1-bb37-47cb-8ff6-4382685270f3 None admin] RBAC: Authorizing `identity:get_user(user_id=63f02a13b768446ca706d7594e04d3fc)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:38.598674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929ebe1-bb37-47cb-8ff6-4382685270f3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:38.600407 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 9/17] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:22:38 2022] GET /identity/v3/users/63f02a13b768446ca706d7594e04d3fc => generated 247 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:22:38.605604 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929ebe1-bb37-47cb-8ff6-4382685270f3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:38.620556 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:38.624273 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095db68-917a-437e-8fb4-bc2643a3249d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:38.625246 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095db68-917a-437e-8fb4-bc2643a3249d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:38.625527 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095db68-917a-437e-8fb4-bc2643a3249d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:38.625801 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095db68-917a-437e-8fb4-bc2643a3249d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:38.644086 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0095db68-917a-437e-8fb4-bc2643a3249d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:38.645572 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0095db68-917a-437e-8fb4-bc2643a3249d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:38.663784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-0095db68-917a-437e-8fb4-bc2643a3249d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:22:38.663784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:22:38.663784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:22:38.663784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:22:38.663784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:22:38.663784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:22:38.663784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:22:38.663784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:22:38.663784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:22:38.663784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:22:38.663784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:22:38.663784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:22:38.663784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:22:38.663784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:22:38.663784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:22:38.663784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:22:38.664746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:22:38.664746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:22:38.664746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:22:38.664746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:22:38.664746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:22:38.664746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:22:38.664746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:22:38.664746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:22:38.664746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:22:38.664746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:22:38.664746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:22:38.664746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:22:38.664746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:22:38.664746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:22:38.664746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:22:38.664746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:22:38.664746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:22:38.665730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:22:38.665730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:22:38.665730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:22:38.665730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 9/18] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:22:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:22:38.670778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095db68-917a-437e-8fb4-bc2643a3249d None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:38.708570 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929ebe1-bb37-47cb-8ff6-4382685270f3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:38.725401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:38.743324 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc11478-4c7b-412a-8bb7-a41eaa6fc35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:38.744122 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc11478-4c7b-412a-8bb7-a41eaa6fc35e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:38.744230 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc11478-4c7b-412a-8bb7-a41eaa6fc35e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:38.744511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc11478-4c7b-412a-8bb7-a41eaa6fc35e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:38.745679 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc11478-4c7b-412a-8bb7-a41eaa6fc35e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:38.747126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc11478-4c7b-412a-8bb7-a41eaa6fc35e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:38.748678 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 10/19] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:22:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:22:38.754414 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc11478-4c7b-412a-8bb7-a41eaa6fc35e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:38.771261 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095db68-917a-437e-8fb4-bc2643a3249d None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:38.804464 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd72fc-ff46-4837-9236-faa66efcefe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:38.805084 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd72fc-ff46-4837-9236-faa66efcefe1 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:38.805384 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd72fc-ff46-4837-9236-faa66efcefe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:38.805641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd72fc-ff46-4837-9236-faa66efcefe1 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:38.806290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd72fc-ff46-4837-9236-faa66efcefe1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:22:38.806592 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd72fc-ff46-4837-9236-faa66efcefe1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:38.808000 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd72fc-ff46-4837-9236-faa66efcefe1 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:38.824705 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 10/20] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:22:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:22:38.834812 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:38.859451 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc11478-4c7b-412a-8bb7-a41eaa6fc35e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:38.897705 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38928227-456e-4a67-a450-a68c8ee88f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:38.898326 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38928227-456e-4a67-a450-a68c8ee88f8a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:38.898605 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38928227-456e-4a67-a450-a68c8ee88f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:38.898875 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38928227-456e-4a67-a450-a68c8ee88f8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:38.899620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38928227-456e-4a67-a450-a68c8ee88f8a None admin] RBAC: Adding query filter params (role.id=c1e514a4b5424f959b069f36cbfd8147, scope.domain.id=default, user.id=63f02a13b768446ca706d7594e04d3fc) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:22:38.899925 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38928227-456e-4a67-a450-a68c8ee88f8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:38.901412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38928227-456e-4a67-a450-a68c8ee88f8a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:38.909641 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 11/21] 10.222.0.33 () {56 vars in 1322 bytes} [Mon Aug 8 08:22:38 2022] GET /identity/v3/role_assignments?user.id=63f02a13b768446ca706d7594e04d3fc&scope.domain.id=default&role.id=c1e514a4b5424f959b069f36cbfd8147 => generated 233 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:22:41.069902 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449d7f9-06c4-4986-9f04-d6a944ea5e5d None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:41.070919 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449d7f9-06c4-4986-9f04-d6a944ea5e5d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:41.071422 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449d7f9-06c4-4986-9f04-d6a944ea5e5d None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:41.072970 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 11/22] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:22:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:22:41.078677 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586fd3b-8a72-4276-b07d-c75cc61a18d1 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:41.079812 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586fd3b-8a72-4276-b07d-c75cc61a18d1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:41.080299 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586fd3b-8a72-4276-b07d-c75cc61a18d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:41.110394 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8586fd3b-8a72-4276-b07d-c75cc61a18d1 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:22:41.111946 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8586fd3b-8a72-4276-b07d-c75cc61a18d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:22:41.195421 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 12/23] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:22:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:22:41.230302 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449d7f9-06c4-4986-9f04-d6a944ea5e5d None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:41.305979 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce08222-e47b-4d7d-87bd-cdb88421a19a None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:41.306598 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce08222-e47b-4d7d-87bd-cdb88421a19a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:41.307091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce08222-e47b-4d7d-87bd-cdb88421a19a None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:41.308887 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 13/24] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:22:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:22:41.313941 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a265e-b153-4e73-a437-a7962574eb34 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:41.314449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a265e-b153-4e73-a437-a7962574eb34 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:41.314931 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a265e-b153-4e73-a437-a7962574eb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:41.350277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-902a265e-b153-4e73-a437-a7962574eb34 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:22:41.351811 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-902a265e-b153-4e73-a437-a7962574eb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:22:41.425851 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449d7f9-06c4-4986-9f04-d6a944ea5e5d None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:41.459017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 14/25] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:22:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:22:41.474240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:41.498895 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902a265e-b153-4e73-a437-a7962574eb34 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:41.502929 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:41.509181 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b69d4e-9f10-42b9-89f2-18e67f43aca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:41.509181 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b69d4e-9f10-42b9-89f2-18e67f43aca6 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:41.509181 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b69d4e-9f10-42b9-89f2-18e67f43aca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:41.509181 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b69d4e-9f10-42b9-89f2-18e67f43aca6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:41.534074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b69d4e-9f10-42b9-89f2-18e67f43aca6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:22:41.541117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-12b69d4e-9f10-42b9-89f2-18e67f43aca6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:22:41.541117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:22:41.541117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:22:41.541117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:22:41.541117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:22:41.541117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:22:41.541117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:22:41.541117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:22:41.541117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:22:41.541117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:22:41.541117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:22:41.541117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:22:41.541117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:22:41.541117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:22:41.541117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:22:41.541117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:22:41.542004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:22:41.542004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:22:41.542004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:22:41.542004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:22:41.542004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:22:41.542004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:22:41.542004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:22:41.542004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:22:41.542004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:22:41.542004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:22:41.542004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:22:41.542004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:22:41.542004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:22:41.542004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:22:41.542004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:22:41.542004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:22:41.542752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:22:41.542752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:22:41.542752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:22:41.542752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:22:41.542752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:22:41.542752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:22:41.542752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:22:41.542752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:22:41.542752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:22:41.542752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:22:41.542752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:22:41.542752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:22:41.542752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 12/26] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:22:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:22:41.551077 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b69d4e-9f10-42b9-89f2-18e67f43aca6 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:41.699995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902a265e-b153-4e73-a437-a7962574eb34 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:41.701607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b69d4e-9f10-42b9-89f2-18e67f43aca6 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:41.766895 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24860cad-ad01-47dd-bcb1-0d4c9156dde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:41.767942 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24860cad-ad01-47dd-bcb1-0d4c9156dde9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:41.768443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24860cad-ad01-47dd-bcb1-0d4c9156dde9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:41.768923 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24860cad-ad01-47dd-bcb1-0d4c9156dde9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:41.780825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24860cad-ad01-47dd-bcb1-0d4c9156dde9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:41.782813 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24860cad-ad01-47dd-bcb1-0d4c9156dde9 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:41.801107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-24860cad-ad01-47dd-bcb1-0d4c9156dde9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:22:41.801107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:22:41.801107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:22:41.801107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:22:41.801107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:22:41.801107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:22:41.801107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:22:41.801107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:22:41.801107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:22:41.801107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:22:41.801107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:22:41.801107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:22:41.801107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:22:41.801107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:22:41.801107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:22:41.801107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:22:41.802060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:22:41.802060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:22:41.802060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:22:41.802060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:22:41.802060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:22:41.802060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:22:41.802060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:22:41.802060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:22:41.802060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:22:41.802060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:22:41.802060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:22:41.802060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:22:41.802060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:22:41.802060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:22:41.802060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:22:41.802060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:22:41.802060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:22:41.802891 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:22:41.802891 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:22:41.802891 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:22:41.805199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 15/27] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:22:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:22:41.806707 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32f254-5734-42c5-956f-936ea20c4bdb None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:41.807215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32f254-5734-42c5-956f-936ea20c4bdb None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:41.807683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32f254-5734-42c5-956f-936ea20c4bdb None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:41.809451 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 16/28] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:22:41.810726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a2c9c2-b278-4078-bdf0-ce6ce32554c7 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:41.811222 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a2c9c2-b278-4078-bdf0-ce6ce32554c7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:41.811876 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a2c9c2-b278-4078-bdf0-ce6ce32554c7 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:41.819509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 17/29] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:22:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:22:41.819509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a2c9c2-b278-4078-bdf0-ce6ce32554c7 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:41.856643 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97532526-efa2-436f-8d99-f080cfcb0867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:41.857280 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97532526-efa2-436f-8d99-f080cfcb0867 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:41.857905 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97532526-efa2-436f-8d99-f080cfcb0867 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:41.859698 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97532526-efa2-436f-8d99-f080cfcb0867 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:41.860544 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97532526-efa2-436f-8d99-f080cfcb0867 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:22:41.860998 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97532526-efa2-436f-8d99-f080cfcb0867 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:41.862550 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97532526-efa2-436f-8d99-f080cfcb0867 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:41.879797 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 13/30] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:22:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:22:41.881370 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530db59-23f8-4a86-8d6d-c99383e0df96 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:41.881796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530db59-23f8-4a86-8d6d-c99383e0df96 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:41.882204 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530db59-23f8-4a86-8d6d-c99383e0df96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:41.907010 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-8530db59-23f8-4a86-8d6d-c99383e0df96 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:22:41.908866 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8530db59-23f8-4a86-8d6d-c99383e0df96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:22:41.917578 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:41.935125 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a2c9c2-b278-4078-bdf0-ce6ce32554c7 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:41.947441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:41.973484 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919b09c-aaae-413f-a558-41c08b7be006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:41.974520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919b09c-aaae-413f-a558-41c08b7be006 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:41.975059 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919b09c-aaae-413f-a558-41c08b7be006 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:41.975620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919b09c-aaae-413f-a558-41c08b7be006 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:41.976492 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0919b09c-aaae-413f-a558-41c08b7be006 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:22:41.977019 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0919b09c-aaae-413f-a558-41c08b7be006 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:41.978883 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0919b09c-aaae-413f-a558-41c08b7be006 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:41.992963 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 14/31] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:22:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:22:41.994054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23babd3-ea30-464b-b838-5624ce441ce7 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:41.994341 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23babd3-ea30-464b-b838-5624ce441ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:41.994614 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23babd3-ea30-464b-b838-5624ce441ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:42.006615 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 18/32] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:22:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:42.007834 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919b09c-aaae-413f-a558-41c08b7be006 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:42.021184 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-a23babd3-ea30-464b-b838-5624ce441ce7 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:22:42.021184 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a23babd3-ea30-464b-b838-5624ce441ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:22:42.095316 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 15/33] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:22:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:22:42.096949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23babd3-ea30-464b-b838-5624ce441ce7 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:42.142150 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919b09c-aaae-413f-a558-41c08b7be006 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:42.165561 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:42.178791 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962981bc-38ca-45e4-8d72-79ef7e89a620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:42.179435 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962981bc-38ca-45e4-8d72-79ef7e89a620 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:42.179713 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962981bc-38ca-45e4-8d72-79ef7e89a620 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:42.180032 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962981bc-38ca-45e4-8d72-79ef7e89a620 None admin] PATH_INFO: `/v3/users/63f02a13b768446ca706d7594e04d3fc` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:42.181281 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962981bc-38ca-45e4-8d72-79ef7e89a620 None admin] RBAC: Authorizing `identity:get_user(user_id=63f02a13b768446ca706d7594e04d3fc)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:42.189912 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962981bc-38ca-45e4-8d72-79ef7e89a620 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:42.189912 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 19/34] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:22:42 2022] GET /identity/v3/users/63f02a13b768446ca706d7594e04d3fc => generated 247 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:42.189912 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962981bc-38ca-45e4-8d72-79ef7e89a620 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:42.207103 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:42.211871 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23babd3-ea30-464b-b838-5624ce441ce7 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:42.248598 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d2ba2-ad84-4ae8-b9c1-5e30e46c051d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:42.250110 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d2ba2-ad84-4ae8-b9c1-5e30e46c051d None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:42.250587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d2ba2-ad84-4ae8-b9c1-5e30e46c051d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:42.251137 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d2ba2-ad84-4ae8-b9c1-5e30e46c051d None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:42.252214 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d2ba2-ad84-4ae8-b9c1-5e30e46c051d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:42.253916 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d2ba2-ad84-4ae8-b9c1-5e30e46c051d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:42.296122 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962981bc-38ca-45e4-8d72-79ef7e89a620 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:42.332127 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea76c6b-ff99-42d4-90cf-66aa5a256e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:42.332742 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea76c6b-ff99-42d4-90cf-66aa5a256e89 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:42.333022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea76c6b-ff99-42d4-90cf-66aa5a256e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:42.333326 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea76c6b-ff99-42d4-90cf-66aa5a256e89 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:42.333988 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea76c6b-ff99-42d4-90cf-66aa5a256e89 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:42.335388 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea76c6b-ff99-42d4-90cf-66aa5a256e89 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:42.473791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 16/35] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Aug 8 08:22:42 2022] POST /identity/v3/projects => generated 301 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:22:42.475201 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d2ba2-ad84-4ae8-b9c1-5e30e46c051d None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:42.593126 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d2ba2-ad84-4ae8-b9c1-5e30e46c051d None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:42.617789 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 20/36] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Aug 8 08:22:42 2022] POST /identity/v3/roles => generated 219 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:22:42.618233 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea76c6b-ff99-42d4-90cf-66aa5a256e89 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:42.658310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63cc53-1170-4bb5-af94-7c4bd93e8b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:42.658962 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63cc53-1170-4bb5-af94-7c4bd93e8b91 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:42.659247 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63cc53-1170-4bb5-af94-7c4bd93e8b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:42.659545 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63cc53-1170-4bb5-af94-7c4bd93e8b91 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:42.660296 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e63cc53-1170-4bb5-af94-7c4bd93e8b91 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:42.661940 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e63cc53-1170-4bb5-af94-7c4bd93e8b91 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:42.706175 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:42.743230 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 17/37] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Aug 8 08:22:42 2022] POST /identity/v3/roles => generated 225 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:22:42.744679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a5b18-f82e-43b4-9996-bc709a50d4aa None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:42.744973 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a5b18-f82e-43b4-9996-bc709a50d4aa None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:42.745314 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a5b18-f82e-43b4-9996-bc709a50d4aa None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:42.747306 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 18/38] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:22:42.753700 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73c810-13b5-40d2-b942-1156f7b4094e None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:42.754036 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73c810-13b5-40d2-b942-1156f7b4094e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:42.754272 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73c810-13b5-40d2-b942-1156f7b4094e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:42.782346 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-0c73c810-13b5-40d2-b942-1156f7b4094e None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:22:42.783943 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0c73c810-13b5-40d2-b942-1156f7b4094e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:22:42.790798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:42.803062 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea76c6b-ff99-42d4-90cf-66aa5a256e89 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:42.861446 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6cb243-b35c-4d3a-ba2f-d7cda0e5f061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:42.862045 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cb243-b35c-4d3a-ba2f-d7cda0e5f061 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:42.862330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cb243-b35c-4d3a-ba2f-d7cda0e5f061 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:42.862627 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cb243-b35c-4d3a-ba2f-d7cda0e5f061 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:42.864175 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6cb243-b35c-4d3a-ba2f-d7cda0e5f061 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:42.865854 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6cb243-b35c-4d3a-ba2f-d7cda0e5f061 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:42.868604 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 21/39] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:22:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:42.873979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 19/40] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:22:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:22:42.882262 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6cb243-b35c-4d3a-ba2f-d7cda0e5f061 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:42.882850 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d13998-eea2-4277-baf9-71fdade2f29c None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:42.883118 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d13998-eea2-4277-baf9-71fdade2f29c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:42.883383 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d13998-eea2-4277-baf9-71fdade2f29c None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:42.884900 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 20/41] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:22:42.895508 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123ec31-4afe-4d7a-89cf-6050ddea4ac0 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:42.896353 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123ec31-4afe-4d7a-89cf-6050ddea4ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:42.897029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123ec31-4afe-4d7a-89cf-6050ddea4ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:42.930997 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-6123ec31-4afe-4d7a-89cf-6050ddea4ac0 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:22:42.932955 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6123ec31-4afe-4d7a-89cf-6050ddea4ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:22:43.011847 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6cb243-b35c-4d3a-ba2f-d7cda0e5f061 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:43.022411 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 21/42] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:22:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:22:43.023601 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123ec31-4afe-4d7a-89cf-6050ddea4ac0 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:43.026563 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:43.047440 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f6a3e-8c86-477e-8f74-6df09be65c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:43.048106 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f6a3e-8c86-477e-8f74-6df09be65c53 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:43.048386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f6a3e-8c86-477e-8f74-6df09be65c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:43.048667 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f6a3e-8c86-477e-8f74-6df09be65c53 None admin] PATH_INFO: `/v3/domains/default/users/63f02a13b768446ca706d7594e04d3fc/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:43.050745 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f6a3e-8c86-477e-8f74-6df09be65c53 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=63f02a13b768446ca706d7594e04d3fc, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:43.052213 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f6a3e-8c86-477e-8f74-6df09be65c53 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:43.104284 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:43.134474 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123ec31-4afe-4d7a-89cf-6050ddea4ac0 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:43.146942 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 22/43] 10.222.0.33 () {58 vars in 1361 bytes} [Mon Aug 8 08:22:42 2022] PUT /identity/v3/domains/default/users/63f02a13b768446ca706d7594e04d3fc/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:22:43.147914 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f6a3e-8c86-477e-8f74-6df09be65c53 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:43.171630 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb6ed1-54c1-49f3-844f-b10b981186bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:43.172275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb6ed1-54c1-49f3-844f-b10b981186bf None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:43.172557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb6ed1-54c1-49f3-844f-b10b981186bf None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:43.172841 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb6ed1-54c1-49f3-844f-b10b981186bf None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:43.174028 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbb6ed1-54c1-49f3-844f-b10b981186bf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:43.175828 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbb6ed1-54c1-49f3-844f-b10b981186bf None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:43.267595 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 22/44] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Aug 8 08:22:43 2022] POST /identity/v3/roles => generated 223 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:22:43.272029 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f6a3e-8c86-477e-8f74-6df09be65c53 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:43.316146 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13894034-4be2-43c7-acdf-313814ac598f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:43.316858 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13894034-4be2-43c7-acdf-313814ac598f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:43.317172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13894034-4be2-43c7-acdf-313814ac598f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:43.317501 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13894034-4be2-43c7-acdf-313814ac598f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:43.318668 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13894034-4be2-43c7-acdf-313814ac598f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:43.320213 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13894034-4be2-43c7-acdf-313814ac598f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:43.322156 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 23/45] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:22:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:43.328081 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb6ed1-54c1-49f3-844f-b10b981186bf None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:43.444068 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb6ed1-54c1-49f3-844f-b10b981186bf None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:43.473928 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:43.480092 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53792af2-11d7-42f0-8dc3-5d756e551e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:43.480791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53792af2-11d7-42f0-8dc3-5d756e551e3f None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:43.481079 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53792af2-11d7-42f0-8dc3-5d756e551e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:43.481398 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53792af2-11d7-42f0-8dc3-5d756e551e3f None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:43.482053 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53792af2-11d7-42f0-8dc3-5d756e551e3f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:43.484000 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53792af2-11d7-42f0-8dc3-5d756e551e3f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:43.571219 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 23/46] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Aug 8 08:22:43 2022] POST /identity/v3/projects => generated 312 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:22:45.234148 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838b378-9f38-4929-b47b-9535675423d1 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:45.234662 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838b378-9f38-4929-b47b-9535675423d1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:45.234743 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838b378-9f38-4929-b47b-9535675423d1 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:45.236011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 24/47] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:22:45.241850 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0726412-ff87-418a-bf24-3261691a3c0f None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:45.242265 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0726412-ff87-418a-bf24-3261691a3c0f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:45.242403 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0726412-ff87-418a-bf24-3261691a3c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:45.259882 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-e0726412-ff87-418a-bf24-3261691a3c0f None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:22:45.261061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e0726412-ff87-418a-bf24-3261691a3c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:22:45.326940 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 24/48] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:22:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:22:45.350303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838b378-9f38-4929-b47b-9535675423d1 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:45.394271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:45.457500 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838b378-9f38-4929-b47b-9535675423d1 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:45.490362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88fe8cc-93e4-4613-916c-a970f1e07e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:45.490919 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88fe8cc-93e4-4613-916c-a970f1e07e9e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:45.491201 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88fe8cc-93e4-4613-916c-a970f1e07e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:45.491473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88fe8cc-93e4-4613-916c-a970f1e07e9e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:45.501638 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88fe8cc-93e4-4613-916c-a970f1e07e9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:22:45.506729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-b88fe8cc-93e4-4613-916c-a970f1e07e9e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:22:45.506729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:22:45.506729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:22:45.506729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:22:45.506729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:22:45.506729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:22:45.506729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:22:45.506729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:22:45.506729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:22:45.506729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:22:45.506729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:22:45.506729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:22:45.506729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:22:45.506729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:22:45.506729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:22:45.506729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:22:45.507559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:22:45.507559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:22:45.507559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:22:45.507559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:22:45.507559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:22:45.507559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:22:45.507559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:22:45.507559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:22:45.507559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:22:45.507559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:22:45.507559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:22:45.507559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:22:45.507559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:22:45.507559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:22:45.507559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:22:45.507559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:22:45.508303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:22:45.508303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:22:45.508303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:22:45.508303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:22:45.508303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:22:45.508303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:22:45.508303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:22:45.508303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:22:45.508303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:22:45.508303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:22:45.508303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:22:45.508303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:22:45.508303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 25/49] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:22:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:22:45.513842 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0726412-ff87-418a-bf24-3261691a3c0f None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:45.610360 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0726412-ff87-418a-bf24-3261691a3c0f None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:45.623034 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:45.643865 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b564c5-df8e-44bf-95ba-50a464764488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:45.644574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b564c5-df8e-44bf-95ba-50a464764488 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:45.644858 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b564c5-df8e-44bf-95ba-50a464764488 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:45.645177 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b564c5-df8e-44bf-95ba-50a464764488 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:45.645849 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b564c5-df8e-44bf-95ba-50a464764488 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:22:45.646154 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b564c5-df8e-44bf-95ba-50a464764488 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:45.647536 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b564c5-df8e-44bf-95ba-50a464764488 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:45.660708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 25/50] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:22:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:45.661948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88fe8cc-93e4-4613-916c-a970f1e07e9e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:45.705664 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96015632-0a47-4f7f-a07e-e741456d360f None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:45.705952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96015632-0a47-4f7f-a07e-e741456d360f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:45.706230 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96015632-0a47-4f7f-a07e-e741456d360f None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:45.707575 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 26/51] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 08:22:45.712365 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0941213-13ba-446c-8a5e-1269392d9a7d None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:45.712652 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0941213-13ba-446c-8a5e-1269392d9a7d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:45.712929 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0941213-13ba-446c-8a5e-1269392d9a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:45.731458 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-a0941213-13ba-446c-8a5e-1269392d9a7d None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:22:45.732652 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a0941213-13ba-446c-8a5e-1269392d9a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:22:45.760548 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:45.763368 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88fe8cc-93e4-4613-916c-a970f1e07e9e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:45.797302 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0446f-ceb7-4d29-9a1a-1c713ea86e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:45.797935 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0446f-ceb7-4d29-9a1a-1c713ea86e30 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:45.798221 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0446f-ceb7-4d29-9a1a-1c713ea86e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:45.798500 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0446f-ceb7-4d29-9a1a-1c713ea86e30 None admin] PATH_INFO: `/v3/users/63f02a13b768446ca706d7594e04d3fc` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:45.799101 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 27/52] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:22:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:22:45.799897 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0446f-ceb7-4d29-9a1a-1c713ea86e30 None admin] RBAC: Authorizing `identity:get_user(user_id=63f02a13b768446ca706d7594e04d3fc)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:45.801406 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0446f-ceb7-4d29-9a1a-1c713ea86e30 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:45.802974 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 26/53] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:22:45 2022] GET /identity/v3/users/63f02a13b768446ca706d7594e04d3fc => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:45.808407 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0941213-13ba-446c-8a5e-1269392d9a7d None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:45.822007 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0446f-ceb7-4d29-9a1a-1c713ea86e30 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:45.910288 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0941213-13ba-446c-8a5e-1269392d9a7d None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:45.912542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:45.919677 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0446f-ceb7-4d29-9a1a-1c713ea86e30 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:45.944844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501eb12d-03c2-4907-9963-5acdff1678c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:45.945533 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501eb12d-03c2-4907-9963-5acdff1678c0 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:45.945840 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501eb12d-03c2-4907-9963-5acdff1678c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:45.946459 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501eb12d-03c2-4907-9963-5acdff1678c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:45.947820 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501eb12d-03c2-4907-9963-5acdff1678c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:45.949584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501eb12d-03c2-4907-9963-5acdff1678c0 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:45.951143 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 28/54] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:22:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:22:45.955216 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d305747-2416-4932-8da9-5e63deb02e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:45.955844 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d305747-2416-4932-8da9-5e63deb02e5f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:45.956125 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d305747-2416-4932-8da9-5e63deb02e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:45.956456 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d305747-2416-4932-8da9-5e63deb02e5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:45.957609 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d305747-2416-4932-8da9-5e63deb02e5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:45.957709 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501eb12d-03c2-4907-9963-5acdff1678c0 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:45.959105 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d305747-2416-4932-8da9-5e63deb02e5f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:45.960645 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 27/55] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:22:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:22:45.966346 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d305747-2416-4932-8da9-5e63deb02e5f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:46.000245 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:46.054951 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501eb12d-03c2-4907-9963-5acdff1678c0 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:46.064083 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d305747-2416-4932-8da9-5e63deb02e5f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:46.088399 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc5398-4a7c-440f-8e80-473d5f4ffe9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:46.089024 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc5398-4a7c-440f-8e80-473d5f4ffe9a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:46.089337 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc5398-4a7c-440f-8e80-473d5f4ffe9a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:46.089606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc5398-4a7c-440f-8e80-473d5f4ffe9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:46.090322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc5398-4a7c-440f-8e80-473d5f4ffe9a None admin] RBAC: Adding query filter params (role.id=c1e514a4b5424f959b069f36cbfd8147, scope.domain.id=default, user.id=63f02a13b768446ca706d7594e04d3fc) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:22:46.090626 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc5398-4a7c-440f-8e80-473d5f4ffe9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:46.092063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc5398-4a7c-440f-8e80-473d5f4ffe9a None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:46.097450 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71667cfb-a4d0-4b35-8c93-2e1026500062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:46.098055 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71667cfb-a4d0-4b35-8c93-2e1026500062 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:46.098335 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71667cfb-a4d0-4b35-8c93-2e1026500062 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:46.098606 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71667cfb-a4d0-4b35-8c93-2e1026500062 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:46.099245 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71667cfb-a4d0-4b35-8c93-2e1026500062 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:46.100716 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71667cfb-a4d0-4b35-8c93-2e1026500062 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:46.101320 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 29/56] 10.222.0.33 () {56 vars in 1322 bytes} [Mon Aug 8 08:22:45 2022] GET /identity/v3/role_assignments?user.id=63f02a13b768446ca706d7594e04d3fc&scope.domain.id=default&role.id=c1e514a4b5424f959b069f36cbfd8147 => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:22:46.172346 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 28/57] 10.222.0.33 () {60 vars in 1140 bytes} [Mon Aug 8 08:22:45 2022] POST /identity/v3/projects => generated 298 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:22:48.183250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc46fc-a536-492f-8cc8-c8d33dc7cf1a None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:48.183759 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc46fc-a536-492f-8cc8-c8d33dc7cf1a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:48.183841 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc46fc-a536-492f-8cc8-c8d33dc7cf1a None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:48.185137 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 30/58] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:22:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:22:48.190902 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e579b6f-c357-484d-ae90-f8d88893e846 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:48.191297 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e579b6f-c357-484d-ae90-f8d88893e846 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:48.191445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e579b6f-c357-484d-ae90-f8d88893e846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:48.210042 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4e579b6f-c357-484d-ae90-f8d88893e846 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:22:48.211525 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4e579b6f-c357-484d-ae90-f8d88893e846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:22:48.279621 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 29/59] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:22:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:22:48.302978 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc46fc-a536-492f-8cc8-c8d33dc7cf1a None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:48.339406 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:48.403086 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc46fc-a536-492f-8cc8-c8d33dc7cf1a None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:48.438351 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd11f1e-2726-4ac5-92e7-34d1bcbdc95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:48.438922 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd11f1e-2726-4ac5-92e7-34d1bcbdc95e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:48.439205 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd11f1e-2726-4ac5-92e7-34d1bcbdc95e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:48.439505 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd11f1e-2726-4ac5-92e7-34d1bcbdc95e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:48.440613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd11f1e-2726-4ac5-92e7-34d1bcbdc95e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:48.442093 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd11f1e-2726-4ac5-92e7-34d1bcbdc95e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:48.443661 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 31/60] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:22:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:48.449285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e579b6f-c357-484d-ae90-f8d88893e846 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:48.467080 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:48.545216 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e579b6f-c357-484d-ae90-f8d88893e846 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:48.579302 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d9b16-9512-414f-bf9e-94778a0236d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:48.580159 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d9b16-9512-414f-bf9e-94778a0236d4 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:48.580437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d9b16-9512-414f-bf9e-94778a0236d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:48.580707 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d9b16-9512-414f-bf9e-94778a0236d4 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:48.581367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38d9b16-9512-414f-bf9e-94778a0236d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:48.582841 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38d9b16-9512-414f-bf9e-94778a0236d4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:48.672526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 30/61] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 08:22:48 2022] POST /identity/v3/users => generated 275 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:22:51.840159 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d781cbc-994c-47b8-9766-d309e0303aab None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:51.840994 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d781cbc-994c-47b8-9766-d309e0303aab None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:51.841447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d781cbc-994c-47b8-9766-d309e0303aab None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:51.843193 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 32/62] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:22:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:22:51.848766 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559f555-8d5a-4614-b2bc-fce4953da310 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:51.849536 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559f555-8d5a-4614-b2bc-fce4953da310 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:51.849960 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559f555-8d5a-4614-b2bc-fce4953da310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:51.873076 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b559f555-8d5a-4614-b2bc-fce4953da310 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:22:51.874536 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b559f555-8d5a-4614-b2bc-fce4953da310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:22:51.960761 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 31/63] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:22:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:22:51.987848 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c39247-bebf-4cac-915a-980d5292c3d8 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:51.988359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c39247-bebf-4cac-915a-980d5292c3d8 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:51.988806 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c39247-bebf-4cac-915a-980d5292c3d8 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:51.990676 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 33/64] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:22:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:22:51.998298 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c39247-bebf-4cac-915a-980d5292c3d8 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:52.001270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9358ab2-9c4e-4ce6-9df9-43da0817db7e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:52.001776 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9358ab2-9c4e-4ce6-9df9-43da0817db7e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:52.002269 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9358ab2-9c4e-4ce6-9df9-43da0817db7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:52.013509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:52.026013 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e9358ab2-9c4e-4ce6-9df9-43da0817db7e None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:22:52.027219 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e9358ab2-9c4e-4ce6-9df9-43da0817db7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:22:52.132988 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 32/65] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:22:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:22:52.160346 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9358ab2-9c4e-4ce6-9df9-43da0817db7e None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:52.221313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c39247-bebf-4cac-915a-980d5292c3d8 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:52.226140 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:52.262060 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14086b69-d8fe-4b77-8692-f8aa958a1112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:52.262689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14086b69-d8fe-4b77-8692-f8aa958a1112 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:52.263160 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14086b69-d8fe-4b77-8692-f8aa958a1112 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:52.263438 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14086b69-d8fe-4b77-8692-f8aa958a1112 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:52.275109 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14086b69-d8fe-4b77-8692-f8aa958a1112 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:22:52.280355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-14086b69-d8fe-4b77-8692-f8aa958a1112 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:22:52.280355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:22:52.280355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:22:52.280355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:22:52.280355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:22:52.280355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:22:52.280355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:22:52.280355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:22:52.280355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:22:52.280355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:22:52.280355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:22:52.280355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:22:52.280355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:22:52.280355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:22:52.280355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:22:52.280355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:22:52.281979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:22:52.281979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:22:52.281979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:22:52.281979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:22:52.281979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:22:52.281979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:22:52.281979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:22:52.281979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:22:52.281979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:22:52.281979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:22:52.281979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:22:52.281979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:22:52.281979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:22:52.281979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:22:52.281979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:22:52.281979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:22:52.282761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:22:52.282761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:22:52.282761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:22:52.282761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:22:52.282761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:22:52.282761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:22:52.282761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:22:52.282761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:22:52.282761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:22:52.282761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:22:52.282761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:22:52.282761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:22:52.283709 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 34/66] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:22:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:22:52.284708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a65166-d9d7-4955-8c33-fedff957aa8a None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:52.284998 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a65166-d9d7-4955-8c33-fedff957aa8a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:52.285325 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a65166-d9d7-4955-8c33-fedff957aa8a None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:52.287541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 35/67] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:22:52.289010 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67304b9-d833-4c6c-8535-387f386b4912 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:52.289883 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67304b9-d833-4c6c-8535-387f386b4912 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:52.290661 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67304b9-d833-4c6c-8535-387f386b4912 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:52.293648 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 36/68] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:22:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:22:52.294544 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67304b9-d833-4c6c-8535-387f386b4912 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:52.301056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9358ab2-9c4e-4ce6-9df9-43da0817db7e None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:52.338537 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a52d6-1130-4bff-b4d1-99d2491a68c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:52.339176 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a52d6-1130-4bff-b4d1-99d2491a68c1 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:52.339461 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a52d6-1130-4bff-b4d1-99d2491a68c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:52.340143 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a52d6-1130-4bff-b4d1-99d2491a68c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:52.342581 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:52.351365 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765a52d6-1130-4bff-b4d1-99d2491a68c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:22:52.356177 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-765a52d6-1130-4bff-b4d1-99d2491a68c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:22:52.356177 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:22:52.356177 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:22:52.356177 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:22:52.356177 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:22:52.356177 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:22:52.356177 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:22:52.356177 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:22:52.356177 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:22:52.356177 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:22:52.356177 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:22:52.356177 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:22:52.356177 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:22:52.356177 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:22:52.356177 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:22:52.356177 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:22:52.357683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:22:52.357683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:22:52.357683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:22:52.357683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:22:52.357683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:22:52.357683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:22:52.357683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:22:52.357683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:22:52.357683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:22:52.357683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:22:52.357683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:22:52.357683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:22:52.357683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:22:52.357683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:22:52.357683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:22:52.357683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:22:52.358690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:22:52.358690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:22:52.358690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:22:52.358690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:22:52.358690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:22:52.358690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:22:52.358690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:22:52.358690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:22:52.358690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:22:52.358690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:22:52.358690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:22:52.358690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:22:52.358690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 33/69] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:22:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:22:52.360063 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c459ba-6b95-4fe4-a4b3-ce56bda85cc6 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:52.360063 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c459ba-6b95-4fe4-a4b3-ce56bda85cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:52.360287 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c459ba-6b95-4fe4-a4b3-ce56bda85cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:52.377940 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-28c459ba-6b95-4fe4-a4b3-ce56bda85cc6 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:22:52.379118 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-28c459ba-6b95-4fe4-a4b3-ce56bda85cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:22:52.404100 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67304b9-d833-4c6c-8535-387f386b4912 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:52.441026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82841243-8631-4b31-9066-8848b9ec7e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:52.441674 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82841243-8631-4b31-9066-8848b9ec7e15 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:52.441956 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82841243-8631-4b31-9066-8848b9ec7e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:52.442235 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82841243-8631-4b31-9066-8848b9ec7e15 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:52.442904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82841243-8631-4b31-9066-8848b9ec7e15 None admin] RBAC: Adding query filter params (name=member) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:22:52.443210 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82841243-8631-4b31-9066-8848b9ec7e15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:52.444577 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82841243-8631-4b31-9066-8848b9ec7e15 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:52.446376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 34/70] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:22:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:22:52.447614 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205d336-05dc-4f24-8dad-7033f035dbf6 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:52.447900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205d336-05dc-4f24-8dad-7033f035dbf6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:52.448175 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205d336-05dc-4f24-8dad-7033f035dbf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:52.452646 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 37/71] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 08:22:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:52.458999 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63674958-43a8-42c0-aa84-89a59e42b6ef None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:52.459285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63674958-43a8-42c0-aa84-89a59e42b6ef None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:52.459608 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63674958-43a8-42c0-aa84-89a59e42b6ef None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:52.461429 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 38/72] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:22:52.462138 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63674958-43a8-42c0-aa84-89a59e42b6ef None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:52.466441 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-7205d336-05dc-4f24-8dad-7033f035dbf6 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:22:52.467635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7205d336-05dc-4f24-8dad-7033f035dbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:22:52.491649 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:52.535380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 35/73] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:22:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:22:52.536228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205d336-05dc-4f24-8dad-7033f035dbf6 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:52.569020 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63674958-43a8-42c0-aa84-89a59e42b6ef None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:52.601200 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:52.602219 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd44b2-2318-481c-be8c-aee0a35573f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:52.602831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd44b2-2318-481c-be8c-aee0a35573f0 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:52.603116 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd44b2-2318-481c-be8c-aee0a35573f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:52.603395 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd44b2-2318-481c-be8c-aee0a35573f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:52.604062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd44b2-2318-481c-be8c-aee0a35573f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:22:52.604368 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd44b2-2318-481c-be8c-aee0a35573f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:52.605801 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd44b2-2318-481c-be8c-aee0a35573f0 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:52.614961 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 39/74] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:22:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:52.615816 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd44b2-2318-481c-be8c-aee0a35573f0 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:52.645650 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205d336-05dc-4f24-8dad-7033f035dbf6 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:52.679374 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80824278-d4e1-4753-b145-13dace10038e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:52.679943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80824278-d4e1-4753-b145-13dace10038e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:52.680226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80824278-d4e1-4753-b145-13dace10038e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:52.680501 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80824278-d4e1-4753-b145-13dace10038e None admin] PATH_INFO: `/v3/users/21c9b4f481e04483a2d7cd25eb4e9bab` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:52.693780 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80824278-d4e1-4753-b145-13dace10038e None admin] RBAC: Authorizing `identity:get_user(user_id=21c9b4f481e04483a2d7cd25eb4e9bab)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:52.695255 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80824278-d4e1-4753-b145-13dace10038e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:52.696861 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 36/75] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:22:52 2022] GET /identity/v3/users/21c9b4f481e04483a2d7cd25eb4e9bab => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:52.701552 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8a16b-e03c-47c5-a25c-d3af7c4f785e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:52.701552 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8a16b-e03c-47c5-a25c-d3af7c4f785e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:52.701552 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8a16b-e03c-47c5-a25c-d3af7c4f785e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:52.703108 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:52.714599 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd44b2-2318-481c-be8c-aee0a35573f0 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:52.715983 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-76c8a16b-e03c-47c5-a25c-d3af7c4f785e None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:22:52.717303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-76c8a16b-e03c-47c5-a25c-d3af7c4f785e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:22:52.748501 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12e565-5b6a-403a-8e63-06122bdf230a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:52.749134 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12e565-5b6a-403a-8e63-06122bdf230a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:52.749495 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12e565-5b6a-403a-8e63-06122bdf230a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:52.749740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12e565-5b6a-403a-8e63-06122bdf230a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:52.759590 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12e565-5b6a-403a-8e63-06122bdf230a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:22:52.764290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-ef12e565-5b6a-403a-8e63-06122bdf230a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 08:22:52.764290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:22:52.764290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:22:52.764290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:22:52.764290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:22:52.764290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:22:52.764290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:22:52.764290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:22:52.764290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:22:52.764290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:22:52.764290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:22:52.764290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:22:52.764290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:22:52.764290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:22:52.764290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:22:52.764290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:22:52.765303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:22:52.765303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:22:52.765303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:22:52.765303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:22:52.765303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:22:52.765303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:22:52.765303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:22:52.765303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:22:52.765303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:22:52.765303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:22:52.765303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:22:52.765303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:22:52.765303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:22:52.765303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:22:52.765303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:22:52.765303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:22:52.766877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:22:52.766877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:22:52.766877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:22:52.766877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:22:52.766877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:22:52.766877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:22:52.766877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:22:52.766877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:22:52.766877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:22:52.766877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:22:52.766877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 08:22:52.766877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:22:52.766877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 40/76] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 08:22:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:22:52.766877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12e565-5b6a-403a-8e63-06122bdf230a None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:52.787439 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 37/77] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:22:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:22:52.788637 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8a16b-e03c-47c5-a25c-d3af7c4f785e None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:52.849319 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:52.868396 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12e565-5b6a-403a-8e63-06122bdf230a None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:52.901258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a700813-1151-443d-8d9b-f5de3495ee05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:52.901880 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a700813-1151-443d-8d9b-f5de3495ee05 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:52.902157 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a700813-1151-443d-8d9b-f5de3495ee05 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:52.902425 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a700813-1151-443d-8d9b-f5de3495ee05 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:52.903523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a700813-1151-443d-8d9b-f5de3495ee05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:52.904947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a700813-1151-443d-8d9b-f5de3495ee05 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:52.905049 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8a16b-e03c-47c5-a25c-d3af7c4f785e None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:52.907359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 41/78] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:22:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:52.908122 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a700813-1151-443d-8d9b-f5de3495ee05 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:52.928138 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:52.940555 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f0574-e0a5-4bc9-8db1-d8c3b6802c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:52.941130 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f0574-e0a5-4bc9-8db1-d8c3b6802c89 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:52.941428 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f0574-e0a5-4bc9-8db1-d8c3b6802c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:52.941699 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f0574-e0a5-4bc9-8db1-d8c3b6802c89 None admin] PATH_INFO: `/v3/users/63f02a13b768446ca706d7594e04d3fc` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:52.942831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f0574-e0a5-4bc9-8db1-d8c3b6802c89 None admin] RBAC: Authorizing `identity:get_user(user_id=63f02a13b768446ca706d7594e04d3fc)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:52.944266 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f0574-e0a5-4bc9-8db1-d8c3b6802c89 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:52.947413 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 38/79] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:22:52 2022] GET /identity/v3/users/63f02a13b768446ca706d7594e04d3fc => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:52.947413 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f0574-e0a5-4bc9-8db1-d8c3b6802c89 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:53.008043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a700813-1151-443d-8d9b-f5de3495ee05 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:53.040313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea0448-ea64-4a64-bb3d-d8e2972992f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:53.040919 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea0448-ea64-4a64-bb3d-d8e2972992f5 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:53.041224 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea0448-ea64-4a64-bb3d-d8e2972992f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:53.041505 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea0448-ea64-4a64-bb3d-d8e2972992f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:53.043418 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f0574-e0a5-4bc9-8db1-d8c3b6802c89 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:53.055134 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:53.078932 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51212378-4b9a-4896-878e-e704e0ed2c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:53.079785 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51212378-4b9a-4896-878e-e704e0ed2c5b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:53.080086 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51212378-4b9a-4896-878e-e704e0ed2c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:53.080352 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51212378-4b9a-4896-878e-e704e0ed2c5b None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:53.081015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51212378-4b9a-4896-878e-e704e0ed2c5b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:22:53.081346 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51212378-4b9a-4896-878e-e704e0ed2c5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:53.082810 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51212378-4b9a-4896-878e-e704e0ed2c5b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:53.095416 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 39/80] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:22:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:53.095416 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51212378-4b9a-4896-878e-e704e0ed2c5b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:53.112407 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ea0448-ea64-4a64-bb3d-d8e2972992f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:53.113899 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ea0448-ea64-4a64-bb3d-d8e2972992f5 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:53.130882 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:53.189559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51212378-4b9a-4896-878e-e704e0ed2c5b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:53.212122 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 42/81] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:22:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:22:53.212866 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea0448-ea64-4a64-bb3d-d8e2972992f5 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:53.222320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb025ac-4ebc-4b14-87bb-885c4f73d888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:53.223165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb025ac-4ebc-4b14-87bb-885c4f73d888 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:53.223442 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb025ac-4ebc-4b14-87bb-885c4f73d888 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:53.223711 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb025ac-4ebc-4b14-87bb-885c4f73d888 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:53.233485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb025ac-4ebc-4b14-87bb-885c4f73d888 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:22:53.239347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-feb025ac-4ebc-4b14-87bb-885c4f73d888 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:22:53.239347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:22:53.239347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:22:53.239347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:22:53.239347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:22:53.239347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:22:53.239347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:22:53.239347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:22:53.239347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:22:53.239347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:22:53.239347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:22:53.239347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:22:53.239347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:22:53.239347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:22:53.239347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:22:53.239347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:22:53.240320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:22:53.240320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:22:53.240320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:22:53.240320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:22:53.240320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:22:53.240320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:22:53.240320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:22:53.240320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:22:53.240320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:22:53.240320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:22:53.240320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:22:53.240320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:22:53.240320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:22:53.240320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:22:53.240320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:22:53.240320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:22:53.241072 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:22:53.241072 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:22:53.241072 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:22:53.241072 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:22:53.241072 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:22:53.241072 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:22:53.241072 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:22:53.241072 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:22:53.241072 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:22:53.241072 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:22:53.241072 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:22:53.241072 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:22:53.241072 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 40/82] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:22:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:22:53.241743 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb025ac-4ebc-4b14-87bb-885c4f73d888 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:53.277085 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:53.314640 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea0448-ea64-4a64-bb3d-d8e2972992f5 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:53.331197 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:53.345072 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb025ac-4ebc-4b14-87bb-885c4f73d888 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:53.347923 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde1330-df8f-4ee5-9b30-c0cf06826986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:53.348528 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde1330-df8f-4ee5-9b30-c0cf06826986 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:53.348806 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde1330-df8f-4ee5-9b30-c0cf06826986 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:53.349075 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde1330-df8f-4ee5-9b30-c0cf06826986 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:53.349739 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde1330-df8f-4ee5-9b30-c0cf06826986 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:53.351203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde1330-df8f-4ee5-9b30-c0cf06826986 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:53.378140 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22616dee-4652-48bc-ae2d-edc0e5c9aca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:53.378743 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22616dee-4652-48bc-ae2d-edc0e5c9aca6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:53.379022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22616dee-4652-48bc-ae2d-edc0e5c9aca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:53.383229 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22616dee-4652-48bc-ae2d-edc0e5c9aca6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:53.444183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22616dee-4652-48bc-ae2d-edc0e5c9aca6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:53.445682 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22616dee-4652-48bc-ae2d-edc0e5c9aca6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:53.457030 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 43/83] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 08:22:53 2022] POST /identity/v3/users => generated 289 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:22:53.463664 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde1330-df8f-4ee5-9b30-c0cf06826986 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:53.483654 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:53.563561 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 41/84] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:22:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:22:53.564467 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22616dee-4652-48bc-ae2d-edc0e5c9aca6 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:53.582525 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde1330-df8f-4ee5-9b30-c0cf06826986 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:53.626882 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09040589-144a-41ee-8a5d-9b670b438059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:53.627838 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09040589-144a-41ee-8a5d-9b670b438059 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:53.628125 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09040589-144a-41ee-8a5d-9b670b438059 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:53.628403 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09040589-144a-41ee-8a5d-9b670b438059 None admin] PATH_INFO: `/v3/users/21c9b4f481e04483a2d7cd25eb4e9bab` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:53.630488 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09040589-144a-41ee-8a5d-9b670b438059 None admin] RBAC: Authorizing `identity:get_user(user_id=21c9b4f481e04483a2d7cd25eb4e9bab)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:53.631959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09040589-144a-41ee-8a5d-9b670b438059 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:53.633603 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 44/85] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:22:53 2022] GET /identity/v3/users/21c9b4f481e04483a2d7cd25eb4e9bab => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:53.634372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09040589-144a-41ee-8a5d-9b670b438059 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:53.682040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:53.683832 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22616dee-4652-48bc-ae2d-edc0e5c9aca6 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:53.729089 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:53.735763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010ecca-81fb-49c8-b4d4-8758f574ae2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:53.736524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010ecca-81fb-49c8-b4d4-8758f574ae2b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:53.736766 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010ecca-81fb-49c8-b4d4-8758f574ae2b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:53.737048 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010ecca-81fb-49c8-b4d4-8758f574ae2b None admin] PATH_INFO: `/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:53.738450 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2010ecca-81fb-49c8-b4d4-8758f574ae2b None admin] RBAC: Authorizing `identity:get_project(project_id=ce0954e79d28403bb7e0c4f0b78e4394)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:53.739936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2010ecca-81fb-49c8-b4d4-8758f574ae2b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:53.742041 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 42/86] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:22:53 2022] GET /identity/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:53.742838 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010ecca-81fb-49c8-b4d4-8758f574ae2b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:53.766057 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09040589-144a-41ee-8a5d-9b670b438059 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:53.808250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e2482-4b18-4289-ae97-9e3c3ed9d599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:53.808875 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e2482-4b18-4289-ae97-9e3c3ed9d599 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:53.809175 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e2482-4b18-4289-ae97-9e3c3ed9d599 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:53.809452 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e2482-4b18-4289-ae97-9e3c3ed9d599 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:53.810130 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e2482-4b18-4289-ae97-9e3c3ed9d599 None admin] RBAC: Adding query filter params (name=member) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:22:53.810431 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e2482-4b18-4289-ae97-9e3c3ed9d599 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:53.811796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e2482-4b18-4289-ae97-9e3c3ed9d599 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:53.821073 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 45/87] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 08:22:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:53.821946 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e2482-4b18-4289-ae97-9e3c3ed9d599 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:53.873033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010ecca-81fb-49c8-b4d4-8758f574ae2b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:53.918854 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82788337-92f0-4fdf-a586-17b0eb39f0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:53.919745 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82788337-92f0-4fdf-a586-17b0eb39f0ef None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:53.920025 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82788337-92f0-4fdf-a586-17b0eb39f0ef None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:53.920685 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82788337-92f0-4fdf-a586-17b0eb39f0ef None admin] PATH_INFO: `/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:53.922072 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82788337-92f0-4fdf-a586-17b0eb39f0ef None admin] RBAC: Authorizing `identity:get_project(project_id=ce0954e79d28403bb7e0c4f0b78e4394)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:53.923748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82788337-92f0-4fdf-a586-17b0eb39f0ef None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:53.926336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 43/88] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:22:53 2022] GET /identity/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:53.927491 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82788337-92f0-4fdf-a586-17b0eb39f0ef None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:53.938115 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e2482-4b18-4289-ae97-9e3c3ed9d599 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:53.940217 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:53.973965 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7da459-d455-45ca-adba-9d38ecaf96dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:53.977575 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7da459-d455-45ca-adba-9d38ecaf96dc None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:53.977947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7da459-d455-45ca-adba-9d38ecaf96dc None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:53.978238 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7da459-d455-45ca-adba-9d38ecaf96dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:54.042018 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:54.053834 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7da459-d455-45ca-adba-9d38ecaf96dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:54.055320 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7da459-d455-45ca-adba-9d38ecaf96dc None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:54.080167 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82788337-92f0-4fdf-a586-17b0eb39f0ef None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:54.116690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95528728-8899-441a-89f3-8b5931569406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:54.117680 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95528728-8899-441a-89f3-8b5931569406 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:54.118240 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95528728-8899-441a-89f3-8b5931569406 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:54.118710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95528728-8899-441a-89f3-8b5931569406 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:54.119634 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95528728-8899-441a-89f3-8b5931569406 None admin] RBAC: Adding query filter params (role.id=b0ba6e2bf793402abe2e680c2a7dcb53, scope.project.id=ce0954e79d28403bb7e0c4f0b78e4394, user.id=21c9b4f481e04483a2d7cd25eb4e9bab) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:22:54.120162 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95528728-8899-441a-89f3-8b5931569406 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:54.121870 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95528728-8899-441a-89f3-8b5931569406 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:54.128483 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 44/89] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:22:53 2022] GET /identity/v3/role_assignments?user.id=21c9b4f481e04483a2d7cd25eb4e9bab&scope.project.id=ce0954e79d28403bb7e0c4f0b78e4394&role.id=b0ba6e2bf793402abe2e680c2a7dcb53 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:54.129548 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95528728-8899-441a-89f3-8b5931569406 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:54.170131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 46/90] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:22:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:22:54.171034 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7da459-d455-45ca-adba-9d38ecaf96dc None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:54.175916 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:54.248447 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95528728-8899-441a-89f3-8b5931569406 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:54.282216 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:54.288336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be82b44-9065-481a-9ad7-cde334ce1700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:54.288633 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7da459-d455-45ca-adba-9d38ecaf96dc None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:54.289506 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be82b44-9065-481a-9ad7-cde334ce1700 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:54.290044 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be82b44-9065-481a-9ad7-cde334ce1700 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:54.290522 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be82b44-9065-481a-9ad7-cde334ce1700 None admin] PATH_INFO: `/v3/users/21c9b4f481e04483a2d7cd25eb4e9bab` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:54.296748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be82b44-9065-481a-9ad7-cde334ce1700 None admin] RBAC: Authorizing `identity:get_user(user_id=21c9b4f481e04483a2d7cd25eb4e9bab)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:54.296748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be82b44-9065-481a-9ad7-cde334ce1700 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:54.296748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 45/91] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:22:54 2022] GET /identity/v3/users/21c9b4f481e04483a2d7cd25eb4e9bab => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:54.296748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be82b44-9065-481a-9ad7-cde334ce1700 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:54.328884 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54112345-5544-4d1e-b9e6-67cc63c73a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:54.329897 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54112345-5544-4d1e-b9e6-67cc63c73a53 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:54.330187 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54112345-5544-4d1e-b9e6-67cc63c73a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:54.330467 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54112345-5544-4d1e-b9e6-67cc63c73a53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:54.331214 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54112345-5544-4d1e-b9e6-67cc63c73a53 None admin] RBAC: Adding query filter params (role.id=c1e514a4b5424f959b069f36cbfd8147, scope.project.id=ce0954e79d28403bb7e0c4f0b78e4394, user.id=63f02a13b768446ca706d7594e04d3fc) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:22:54.331526 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54112345-5544-4d1e-b9e6-67cc63c73a53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:54.332980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54112345-5544-4d1e-b9e6-67cc63c73a53 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:54.339281 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 47/92] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:22:54 2022] GET /identity/v3/role_assignments?user.id=63f02a13b768446ca706d7594e04d3fc&scope.project.id=ce0954e79d28403bb7e0c4f0b78e4394&role.id=c1e514a4b5424f959b069f36cbfd8147 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:54.340202 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54112345-5544-4d1e-b9e6-67cc63c73a53 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:54.418428 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be82b44-9065-481a-9ad7-cde334ce1700 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:54.459963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b8a8d-7bee-422b-97de-72ebc9a2a82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:54.460990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b8a8d-7bee-422b-97de-72ebc9a2a82b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:54.461530 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b8a8d-7bee-422b-97de-72ebc9a2a82b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:54.462001 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b8a8d-7bee-422b-97de-72ebc9a2a82b None admin] PATH_INFO: `/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:54.463487 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b8a8d-7bee-422b-97de-72ebc9a2a82b None admin] RBAC: Authorizing `identity:get_project(project_id=ce0954e79d28403bb7e0c4f0b78e4394)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:54.465362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b8a8d-7bee-422b-97de-72ebc9a2a82b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:54.467748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 46/93] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:22:54 2022] GET /identity/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:54.474562 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b8a8d-7bee-422b-97de-72ebc9a2a82b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:54.520977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:54.558287 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:54.597257 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b8a8d-7bee-422b-97de-72ebc9a2a82b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:54.631124 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54112345-5544-4d1e-b9e6-67cc63c73a53 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:54.635037 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c983c8b-1c1e-4e9f-aa79-c7bc8af06700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:54.635925 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c983c8b-1c1e-4e9f-aa79-c7bc8af06700 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:54.636435 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c983c8b-1c1e-4e9f-aa79-c7bc8af06700 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:54.636901 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c983c8b-1c1e-4e9f-aa79-c7bc8af06700 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:54.638664 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c983c8b-1c1e-4e9f-aa79-c7bc8af06700 None admin] RBAC: Adding query filter params (role.id=152f4082bb67479ebaeb08c2fa683399, scope.project.id=ce0954e79d28403bb7e0c4f0b78e4394, user.id=21c9b4f481e04483a2d7cd25eb4e9bab) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:22:54.639195 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c983c8b-1c1e-4e9f-aa79-c7bc8af06700 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:54.640937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c983c8b-1c1e-4e9f-aa79-c7bc8af06700 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:54.647696 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 47/94] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:22:54 2022] GET /identity/v3/role_assignments?user.id=21c9b4f481e04483a2d7cd25eb4e9bab&scope.project.id=ce0954e79d28403bb7e0c4f0b78e4394&role.id=152f4082bb67479ebaeb08c2fa683399 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:54.728042 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebc314-7165-4573-8191-843958d8463d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:54.728739 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebc314-7165-4573-8191-843958d8463d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:54.729028 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebc314-7165-4573-8191-843958d8463d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:54.729335 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebc314-7165-4573-8191-843958d8463d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:54.833160 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ebc314-7165-4573-8191-843958d8463d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:54.834661 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ebc314-7165-4573-8191-843958d8463d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:54.920753 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:54.978589 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 48/95] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:22:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:22:54.979208 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c983c8b-1c1e-4e9f-aa79-c7bc8af06700 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:55.123926 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c983c8b-1c1e-4e9f-aa79-c7bc8af06700 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:55.165727 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:55.172163 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc52657-a9a9-4c83-9e1e-e8bf071f6cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:55.172966 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc52657-a9a9-4c83-9e1e-e8bf071f6cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:55.173292 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc52657-a9a9-4c83-9e1e-e8bf071f6cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:55.173572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc52657-a9a9-4c83-9e1e-e8bf071f6cb1 None admin] PATH_INFO: `/v3/projects/16dec4cc338146a9bae9c5887b9b3197` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:55.177968 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc52657-a9a9-4c83-9e1e-e8bf071f6cb1 None admin] RBAC: Authorizing `identity:get_project(project_id=16dec4cc338146a9bae9c5887b9b3197)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:55.179451 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc52657-a9a9-4c83-9e1e-e8bf071f6cb1 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:55.181136 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 48/96] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:22:54 2022] GET /identity/v3/projects/16dec4cc338146a9bae9c5887b9b3197 => generated 312 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:22:55.187764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebc314-7165-4573-8191-843958d8463d None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:55.330555 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebc314-7165-4573-8191-843958d8463d None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:55.375717 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce7ea9-188a-49c9-98eb-2e08ec67d0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:55.376254 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce7ea9-188a-49c9-98eb-2e08ec67d0fa None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:55.376540 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce7ea9-188a-49c9-98eb-2e08ec67d0fa None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:55.376816 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce7ea9-188a-49c9-98eb-2e08ec67d0fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:55.377583 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce7ea9-188a-49c9-98eb-2e08ec67d0fa None admin] RBAC: Adding query filter params (role.id=b0ba6e2bf793402abe2e680c2a7dcb53, scope.project.id=16dec4cc338146a9bae9c5887b9b3197, user.id=21c9b4f481e04483a2d7cd25eb4e9bab) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:22:55.377894 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce7ea9-188a-49c9-98eb-2e08ec67d0fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:55.379321 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce7ea9-188a-49c9-98eb-2e08ec67d0fa None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:55.389363 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 49/97] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:22:55 2022] GET /identity/v3/role_assignments?user.id=21c9b4f481e04483a2d7cd25eb4e9bab&scope.project.id=16dec4cc338146a9bae9c5887b9b3197&role.id=b0ba6e2bf793402abe2e680c2a7dcb53 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:56.055738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9734170-c468-4e41-b3b3-500155fdf45c None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:56.056257 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9734170-c468-4e41-b3b3-500155fdf45c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:56.056415 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9734170-c468-4e41-b3b3-500155fdf45c None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:56.057989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 49/98] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:22:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:22:56.068856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758290e-ed2d-4c16-b369-64f8addbc144 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:56.069396 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758290e-ed2d-4c16-b369-64f8addbc144 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:56.069472 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758290e-ed2d-4c16-b369-64f8addbc144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:56.092201 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-c758290e-ed2d-4c16-b369-64f8addbc144 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:22:56.093490 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c758290e-ed2d-4c16-b369-64f8addbc144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:22:56.128671 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:56.190470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 50/99] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:22:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:22:56.233098 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9734170-c468-4e41-b3b3-500155fdf45c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:56.369837 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9734170-c468-4e41-b3b3-500155fdf45c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:56.416221 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f93aaf-a58a-49c1-bb5b-3512f82d60e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:56.416884 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f93aaf-a58a-49c1-bb5b-3512f82d60e2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:56.421236 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f93aaf-a58a-49c1-bb5b-3512f82d60e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:56.421236 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f93aaf-a58a-49c1-bb5b-3512f82d60e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:56.421957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f93aaf-a58a-49c1-bb5b-3512f82d60e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:56.423559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f93aaf-a58a-49c1-bb5b-3512f82d60e2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:56.425207 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 50/100] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:22:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:56.431800 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758290e-ed2d-4c16-b369-64f8addbc144 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:56.569627 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758290e-ed2d-4c16-b369-64f8addbc144 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:56.621684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc9a7c-b0e9-414a-9c0c-62294e314471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:56.622230 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc9a7c-b0e9-414a-9c0c-62294e314471 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:56.622515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc9a7c-b0e9-414a-9c0c-62294e314471 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:56.622793 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc9a7c-b0e9-414a-9c0c-62294e314471 None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:56.623457 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdc9a7c-b0e9-414a-9c0c-62294e314471 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:56.625556 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdc9a7c-b0e9-414a-9c0c-62294e314471 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:56.672231 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9b276-9be8-4b94-8873-e1d7a5f71e2a None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:56.672553 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9b276-9be8-4b94-8873-e1d7a5f71e2a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:56.672808 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9b276-9be8-4b94-8873-e1d7a5f71e2a None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:56.674324 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 51/101] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:22:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:22:56.679476 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec224e8-06ae-4da3-924b-977bd820e7cd None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:56.679777 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec224e8-06ae-4da3-924b-977bd820e7cd None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:56.680059 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec224e8-06ae-4da3-924b-977bd820e7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:56.701725 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-3ec224e8-06ae-4da3-924b-977bd820e7cd None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:22:56.702966 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3ec224e8-06ae-4da3-924b-977bd820e7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:22:56.722927 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 51/102] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Aug 8 08:22:56 2022] POST /identity/v3/projects => generated 302 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:22:56.798011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:56.807726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 52/103] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:22:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:22:56.843951 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc9a7c-b0e9-414a-9c0c-62294e314471 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:56.861435 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:56.984754 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc9a7c-b0e9-414a-9c0c-62294e314471 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:57.033771 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7ee32-7287-428d-b7f3-0bbd0e35f90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:57.034461 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7ee32-7287-428d-b7f3-0bbd0e35f90c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:57.034754 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7ee32-7287-428d-b7f3-0bbd0e35f90c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:57.035030 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7ee32-7287-428d-b7f3-0bbd0e35f90c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:57.050814 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee7ee32-7287-428d-b7f3-0bbd0e35f90c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:22:57.055740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-6ee7ee32-7287-428d-b7f3-0bbd0e35f90c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 08:22:57.055740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:22:57.055740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:22:57.055740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:22:57.055740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:22:57.055740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:22:57.055740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:22:57.055740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:22:57.055740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:22:57.055740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:22:57.055740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:22:57.055740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:22:57.055740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:22:57.055740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:22:57.055740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:22:57.055740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:22:57.056578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:22:57.056578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:22:57.056578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:22:57.056578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:22:57.056578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:22:57.056578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:22:57.056578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:22:57.056578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:22:57.056578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:22:57.056578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:22:57.056578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:22:57.056578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:22:57.056578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:22:57.056578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:22:57.056578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:22:57.056578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:22:57.058831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:22:57.058831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:22:57.058831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:22:57.058831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:22:57.058831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:22:57.058831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:22:57.058831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:22:57.058831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:22:57.058831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:22:57.058831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:22:57.058831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 08:22:57.058831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:22:57.058831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 52/104] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:22:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:22:57.067112 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec224e8-06ae-4da3-924b-977bd820e7cd None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:57.203090 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec224e8-06ae-4da3-924b-977bd820e7cd None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:57.245830 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bad533-04b3-4ca0-9c66-01bc247b5fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:57.246381 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bad533-04b3-4ca0-9c66-01bc247b5fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:57.246667 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bad533-04b3-4ca0-9c66-01bc247b5fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:57.246942 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bad533-04b3-4ca0-9c66-01bc247b5fd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:57.247682 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bad533-04b3-4ca0-9c66-01bc247b5fd4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:22:57.247990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bad533-04b3-4ca0-9c66-01bc247b5fd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:57.249472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bad533-04b3-4ca0-9c66-01bc247b5fd4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:57.261098 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 53/105] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 08:22:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:57.267343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7ee32-7287-428d-b7f3-0bbd0e35f90c None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:57.282701 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd1681-fb5d-4736-ba2d-37d4d4f0cd89 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:57.282988 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd1681-fb5d-4736-ba2d-37d4d4f0cd89 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:57.283265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd1681-fb5d-4736-ba2d-37d4d4f0cd89 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:57.284754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 54/106] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 08:22:57.290365 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e571f-3dac-42fc-93c3-7ab01c54caad None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:57.290654 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e571f-3dac-42fc-93c3-7ab01c54caad None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:57.290930 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e571f-3dac-42fc-93c3-7ab01c54caad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:57.323620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4b7e571f-3dac-42fc-93c3-7ab01c54caad None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:22:57.324856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4b7e571f-3dac-42fc-93c3-7ab01c54caad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:22:57.364128 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:57.409629 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7ee32-7287-428d-b7f3-0bbd0e35f90c None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:57.414176 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:57.454274 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f28a9-bfa4-4a70-8458-3bfd91734305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:57.455163 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 55/107] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:22:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:22:57.455294 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f28a9-bfa4-4a70-8458-3bfd91734305 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:57.455727 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f28a9-bfa4-4a70-8458-3bfd91734305 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:57.456100 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f28a9-bfa4-4a70-8458-3bfd91734305 None admin] PATH_INFO: `/v3/users/becd3f0974834999b872e1cbbe9a4d6b` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:57.456457 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e1291-5d18-4d5f-8bba-dc69269dd7e3 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:57.456745 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e1291-5d18-4d5f-8bba-dc69269dd7e3 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:57.457022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e1291-5d18-4d5f-8bba-dc69269dd7e3 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:57.458433 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 56/108] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:22:57.463456 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a228c25-b097-408d-a712-fadba48291cb None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:57.463743 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a228c25-b097-408d-a712-fadba48291cb None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:57.464022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a228c25-b097-408d-a712-fadba48291cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:57.487540 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8a228c25-b097-408d-a712-fadba48291cb None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:22:57.488726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8a228c25-b097-408d-a712-fadba48291cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:22:57.492153 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f28a9-bfa4-4a70-8458-3bfd91734305 None admin] RBAC: Authorizing `identity:get_user(user_id=becd3f0974834999b872e1cbbe9a4d6b)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:57.493656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f28a9-bfa4-4a70-8458-3bfd91734305 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:57.495356 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 53/109] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:22:57 2022] GET /identity/v3/users/becd3f0974834999b872e1cbbe9a4d6b => generated 289 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:57.496174 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f28a9-bfa4-4a70-8458-3bfd91734305 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:57.625006 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 57/110] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:22:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:22:57.625923 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a228c25-b097-408d-a712-fadba48291cb None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:57.646428 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f28a9-bfa4-4a70-8458-3bfd91734305 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:57.685511 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:57.691881 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cdfe4c-c42b-4205-884a-b4df45d62755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:57.696885 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdfe4c-c42b-4205-884a-b4df45d62755 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:57.696885 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdfe4c-c42b-4205-884a-b4df45d62755 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:57.696885 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdfe4c-c42b-4205-884a-b4df45d62755 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:57.709068 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cdfe4c-c42b-4205-884a-b4df45d62755 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:22:57.715672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-12cdfe4c-c42b-4205-884a-b4df45d62755 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:22:57.715672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:22:57.715672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:22:57.715672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:22:57.715672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:22:57.715672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:22:57.715672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:22:57.715672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:22:57.715672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:22:57.715672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:22:57.715672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:22:57.715672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:22:57.715672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:22:57.715672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:22:57.715672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:22:57.715672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:22:57.717758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:22:57.717758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:22:57.717758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:22:57.717758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:22:57.717758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:22:57.717758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:22:57.717758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:22:57.717758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:22:57.717758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:22:57.717758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:22:57.717758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:22:57.717758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:22:57.717758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:22:57.717758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:22:57.717758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:22:57.717758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:22:57.719198 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:22:57.719198 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:22:57.719198 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:22:57.719198 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:22:57.719198 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:22:57.719198 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:22:57.719198 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:22:57.719198 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:22:57.719198 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:22:57.719198 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:22:57.719198 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:22:57.719198 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:22:57.719198 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 54/111] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:22:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:22:57.720500 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97897564-7027-4503-9606-5aa2f5ea5614 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:57.720576 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97897564-7027-4503-9606-5aa2f5ea5614 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:57.725379 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97897564-7027-4503-9606-5aa2f5ea5614 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:57.725379 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 55/112] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:22:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:22:57.725379 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97897564-7027-4503-9606-5aa2f5ea5614 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:57.759545 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a228c25-b097-408d-a712-fadba48291cb None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:57.785832 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:57.804108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426361a7-340e-4187-a814-a70e2b1ab2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:57.804788 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426361a7-340e-4187-a814-a70e2b1ab2dd None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:57.805098 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426361a7-340e-4187-a814-a70e2b1ab2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:57.805630 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426361a7-340e-4187-a814-a70e2b1ab2dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:57.860411 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97897564-7027-4503-9606-5aa2f5ea5614 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:57.896198 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984924ac-a1c9-4961-9d07-e81f7b47bb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:57.897200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984924ac-a1c9-4961-9d07-e81f7b47bb16 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:57.897682 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984924ac-a1c9-4961-9d07-e81f7b47bb16 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:57.898150 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984924ac-a1c9-4961-9d07-e81f7b47bb16 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:57.899193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426361a7-340e-4187-a814-a70e2b1ab2dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:57.900956 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426361a7-340e-4187-a814-a70e2b1ab2dd None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:57.910542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984924ac-a1c9-4961-9d07-e81f7b47bb16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:22:57.915564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-984924ac-a1c9-4961-9d07-e81f7b47bb16 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 08:22:57.915564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:22:57.915564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:22:57.915564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:22:57.915564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:22:57.915564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:22:57.915564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:22:57.915564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:22:57.915564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:22:57.915564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:22:57.915564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:22:57.915564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:22:57.915564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:22:57.915564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:22:57.915564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:22:57.915564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:22:57.916388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:22:57.916388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:22:57.916388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:22:57.916388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:22:57.916388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:22:57.916388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:22:57.916388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:22:57.916388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:22:57.916388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:22:57.916388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:22:57.916388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:22:57.916388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:22:57.916388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:22:57.916388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:22:57.916388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:22:57.916388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:22:57.917528 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:22:57.917528 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:22:57.917528 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:22:57.917528 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:22:57.917528 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:22:57.917528 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:22:57.917528 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:22:57.917528 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:22:57.917528 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:22:57.917528 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:22:57.917528 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 08:22:57.917528 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:22:57.919485 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 56/113] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 08:22:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:22:57.925731 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984924ac-a1c9-4961-9d07-e81f7b47bb16 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:58.018963 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:58.020395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 58/114] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:22:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:22:58.021751 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f1521-2386-4352-8253-9024378b600c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:58.022043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f1521-2386-4352-8253-9024378b600c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:58.022320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f1521-2386-4352-8253-9024378b600c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:58.047384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0d0f1521-2386-4352-8253-9024378b600c None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:22:58.049118 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0d0f1521-2386-4352-8253-9024378b600c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:22:58.068423 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:58.077017 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984924ac-a1c9-4961-9d07-e81f7b47bb16 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:58.116461 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490cfde2-1724-4ad0-adfa-d4b14abcf32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:58.117510 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cfde2-1724-4ad0-adfa-d4b14abcf32e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:58.118029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cfde2-1724-4ad0-adfa-d4b14abcf32e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:58.118493 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cfde2-1724-4ad0-adfa-d4b14abcf32e None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:58.119349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490cfde2-1724-4ad0-adfa-d4b14abcf32e None admin] RBAC: Adding query filter params (name=member) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:22:58.119946 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490cfde2-1724-4ad0-adfa-d4b14abcf32e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:58.121537 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490cfde2-1724-4ad0-adfa-d4b14abcf32e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:58.130934 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 57/115] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 08:22:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:58.131975 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490cfde2-1724-4ad0-adfa-d4b14abcf32e None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:58.139632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 59/116] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:22:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:22:58.141361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f1521-2386-4352-8253-9024378b600c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:58.249123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f1521-2386-4352-8253-9024378b600c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:58.264750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490cfde2-1724-4ad0-adfa-d4b14abcf32e None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:58.278692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:58.287856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072f6d5-710c-4b45-a473-c1071f12e715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:58.288784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072f6d5-710c-4b45-a473-c1071f12e715 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:58.289296 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072f6d5-710c-4b45-a473-c1071f12e715 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:58.289767 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072f6d5-710c-4b45-a473-c1071f12e715 None admin] PATH_INFO: `/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:58.291276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072f6d5-710c-4b45-a473-c1071f12e715 None admin] RBAC: Authorizing `identity:get_project(project_id=ce0954e79d28403bb7e0c4f0b78e4394)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:58.292960 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072f6d5-710c-4b45-a473-c1071f12e715 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:58.295081 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 60/117] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:22:58 2022] GET /identity/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:58.296095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072f6d5-710c-4b45-a473-c1071f12e715 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:58.301585 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a669f9-dc9e-4a6b-9467-50791bb0e2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:58.302206 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a669f9-dc9e-4a6b-9467-50791bb0e2ef None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:58.302486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a669f9-dc9e-4a6b-9467-50791bb0e2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:58.302767 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a669f9-dc9e-4a6b-9467-50791bb0e2ef None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:58.303444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a669f9-dc9e-4a6b-9467-50791bb0e2ef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:22:58.303748 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a669f9-dc9e-4a6b-9467-50791bb0e2ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:58.305126 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a669f9-dc9e-4a6b-9467-50791bb0e2ef None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:58.314328 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 58/118] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:22:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:58.318875 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a669f9-dc9e-4a6b-9467-50791bb0e2ef None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:58.323123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:58.408378 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072f6d5-710c-4b45-a473-c1071f12e715 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:58.420892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a669f9-dc9e-4a6b-9467-50791bb0e2ef None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:58.446823 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6516b628-1168-497e-8197-790a7eee7095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:58.447836 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6516b628-1168-497e-8197-790a7eee7095 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:58.448308 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6516b628-1168-497e-8197-790a7eee7095 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:58.448839 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6516b628-1168-497e-8197-790a7eee7095 None admin] PATH_INFO: `/v3/users/21c9b4f481e04483a2d7cd25eb4e9bab` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:58.450483 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6516b628-1168-497e-8197-790a7eee7095 None admin] RBAC: Authorizing `identity:get_user(user_id=21c9b4f481e04483a2d7cd25eb4e9bab)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:58.452143 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6516b628-1168-497e-8197-790a7eee7095 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:58.454645 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 61/119] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:22:58 2022] GET /identity/v3/users/21c9b4f481e04483a2d7cd25eb4e9bab => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:58.455966 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6516b628-1168-497e-8197-790a7eee7095 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:58.462915 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a371f-8eaf-4dc3-be20-529cb3d0f413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:58.463563 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a371f-8eaf-4dc3-be20-529cb3d0f413 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:58.463849 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a371f-8eaf-4dc3-be20-529cb3d0f413 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:58.464127 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a371f-8eaf-4dc3-be20-529cb3d0f413 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:58.475287 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a371f-8eaf-4dc3-be20-529cb3d0f413 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:22:58.480444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-416a371f-8eaf-4dc3-be20-529cb3d0f413 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:22:58.480444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:22:58.480444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:22:58.480444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:22:58.480444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:22:58.480444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:22:58.480444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:22:58.480444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:22:58.480444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:22:58.480444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:22:58.480444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:22:58.480444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:22:58.480444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:22:58.480444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:22:58.480444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:22:58.480444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:22:58.481332 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:22:58.481332 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:22:58.481332 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:22:58.481332 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:22:58.481332 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:22:58.481332 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:22:58.481332 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:22:58.481332 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:22:58.481332 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:22:58.481332 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:22:58.481332 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:22:58.481332 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:22:58.481332 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:22:58.481332 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:22:58.481332 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:22:58.481332 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:22:58.482180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:22:58.482180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:22:58.482180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:22:58.482180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:22:58.482180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:22:58.482180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:22:58.482180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:22:58.482180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:22:58.482180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:22:58.482180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:22:58.482180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:22:58.482180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:22:58.482180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 59/120] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:22:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:22:58.482797 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a371f-8eaf-4dc3-be20-529cb3d0f413 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:58.518320 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:58.556990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:58.565549 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6516b628-1168-497e-8197-790a7eee7095 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:58.588067 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a371f-8eaf-4dc3-be20-529cb3d0f413 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:58.601084 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3f85c-77ff-4803-ac96-83c458c4b425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:58.602479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3f85c-77ff-4803-ac96-83c458c4b425 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:58.602988 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3f85c-77ff-4803-ac96-83c458c4b425 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:58.603617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3f85c-77ff-4803-ac96-83c458c4b425 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:58.604590 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a3f85c-77ff-4803-ac96-83c458c4b425 None admin] RBAC: Adding query filter params (role.id=ab682c3121c04a5fb600dc47750d013c, scope.project.id=ce0954e79d28403bb7e0c4f0b78e4394, user.id=becd3f0974834999b872e1cbbe9a4d6b) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:22:58.605075 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a3f85c-77ff-4803-ac96-83c458c4b425 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:58.606705 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a3f85c-77ff-4803-ac96-83c458c4b425 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:58.613520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 62/121] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:22:58 2022] GET /identity/v3/role_assignments?user.id=becd3f0974834999b872e1cbbe9a4d6b&scope.project.id=ce0954e79d28403bb7e0c4f0b78e4394&role.id=ab682c3121c04a5fb600dc47750d013c => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:58.614610 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3f85c-77ff-4803-ac96-83c458c4b425 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:58.632905 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd8d2d-376b-4251-bd96-83c069ea1e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:58.633940 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd8d2d-376b-4251-bd96-83c069ea1e30 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:58.634731 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd8d2d-376b-4251-bd96-83c069ea1e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:58.635620 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd8d2d-376b-4251-bd96-83c069ea1e30 None admin] PATH_INFO: `/v3/users/21c9b4f481e04483a2d7cd25eb4e9bab` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:58.637135 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bd8d2d-376b-4251-bd96-83c069ea1e30 None admin] RBAC: Authorizing `identity:get_user(user_id=21c9b4f481e04483a2d7cd25eb4e9bab)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:58.639764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bd8d2d-376b-4251-bd96-83c069ea1e30 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:58.642771 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 60/122] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:22:58 2022] GET /identity/v3/users/21c9b4f481e04483a2d7cd25eb4e9bab => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:58.644072 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd8d2d-376b-4251-bd96-83c069ea1e30 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:58.748088 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3f85c-77ff-4803-ac96-83c458c4b425 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:58.767844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd8d2d-376b-4251-bd96-83c069ea1e30 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:58.769772 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:58.791270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a54b88-0ca5-40a1-9f91-2d22455835bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:58.791914 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a54b88-0ca5-40a1-9f91-2d22455835bd None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:58.792202 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a54b88-0ca5-40a1-9f91-2d22455835bd None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:58.792482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a54b88-0ca5-40a1-9f91-2d22455835bd None admin] PATH_INFO: `/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:58.793653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a54b88-0ca5-40a1-9f91-2d22455835bd None admin] RBAC: Authorizing `identity:get_project(project_id=ce0954e79d28403bb7e0c4f0b78e4394)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:58.795223 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a54b88-0ca5-40a1-9f91-2d22455835bd None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:58.797008 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 63/123] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:22:58 2022] GET /identity/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:58.798771 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5818b-ad51-4ecb-8718-0bf475ea5e0c None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:58.804635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5818b-ad51-4ecb-8718-0bf475ea5e0c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:58.804635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5818b-ad51-4ecb-8718-0bf475ea5e0c None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:58.804635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 64/124] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:22:58.805914 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5818b-ad51-4ecb-8718-0bf475ea5e0c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:58.814500 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe1db1f-1bef-4c05-a2eb-20493335e4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:58.815134 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1db1f-1bef-4c05-a2eb-20493335e4eb None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:58.815481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1db1f-1bef-4c05-a2eb-20493335e4eb None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:58.816401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1db1f-1bef-4c05-a2eb-20493335e4eb None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:58.817173 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe1db1f-1bef-4c05-a2eb-20493335e4eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:22:58.817623 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe1db1f-1bef-4c05-a2eb-20493335e4eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:58.820036 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe1db1f-1bef-4c05-a2eb-20493335e4eb None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:58.827198 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:58.833041 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 61/125] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:22:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:58.834631 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d49be-47d9-4ea7-bec6-987c7b06b29f None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:58.835387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d49be-47d9-4ea7-bec6-987c7b06b29f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:58.836098 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d49be-47d9-4ea7-bec6-987c7b06b29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:58.858885 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-141d49be-47d9-4ea7-bec6-987c7b06b29f None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:22:58.862460 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-141d49be-47d9-4ea7-bec6-987c7b06b29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:22:58.926449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5818b-ad51-4ecb-8718-0bf475ea5e0c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:58.952295 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 62/126] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:22:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:22:58.957774 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d49be-47d9-4ea7-bec6-987c7b06b29f None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:58.974957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e9bb3c-a753-4bc4-8fb9-d190e6c4145b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:58.975616 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9bb3c-a753-4bc4-8fb9-d190e6c4145b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:58.975896 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9bb3c-a753-4bc4-8fb9-d190e6c4145b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:58.976170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9bb3c-a753-4bc4-8fb9-d190e6c4145b None admin] PATH_INFO: `/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:58.977801 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e9bb3c-a753-4bc4-8fb9-d190e6c4145b None admin] RBAC: Authorizing `identity:get_project(project_id=ce0954e79d28403bb7e0c4f0b78e4394)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:58.980190 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e9bb3c-a753-4bc4-8fb9-d190e6c4145b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:58.981984 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 65/127] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:22:58 2022] GET /identity/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:58.982797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e9bb3c-a753-4bc4-8fb9-d190e6c4145b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:59.075915 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:59.087424 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:59.094416 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e9bb3c-a753-4bc4-8fb9-d190e6c4145b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:59.095084 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d49be-47d9-4ea7-bec6-987c7b06b29f None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:59.132798 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5951a-4732-45ca-a1e5-d1c7acc5f9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:59.133559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5951a-4732-45ca-a1e5-d1c7acc5f9f9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:59.133851 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5951a-4732-45ca-a1e5-d1c7acc5f9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:59.134182 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5951a-4732-45ca-a1e5-d1c7acc5f9f9 None admin] PATH_INFO: `/v3/users/63f02a13b768446ca706d7594e04d3fc` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:59.135365 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f5951a-4732-45ca-a1e5-d1c7acc5f9f9 None admin] RBAC: Authorizing `identity:get_user(user_id=63f02a13b768446ca706d7594e04d3fc)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:59.137068 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f5951a-4732-45ca-a1e5-d1c7acc5f9f9 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:59.137815 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e633af-c063-46b8-aac7-1227aa275af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:59.138680 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e633af-c063-46b8-aac7-1227aa275af6 None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:59.138796 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 66/128] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:22:58 2022] GET /identity/v3/users/63f02a13b768446ca706d7594e04d3fc => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:59.139254 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e633af-c063-46b8-aac7-1227aa275af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:59.139714 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e633af-c063-46b8-aac7-1227aa275af6 None admin] PATH_INFO: `/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394/users/21c9b4f481e04483a2d7cd25eb4e9bab/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:59.139814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5951a-4732-45ca-a1e5-d1c7acc5f9f9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:59.142043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e633af-c063-46b8-aac7-1227aa275af6 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce0954e79d28403bb7e0c4f0b78e4394, user_id=21c9b4f481e04483a2d7cd25eb4e9bab, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:59.143795 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e633af-c063-46b8-aac7-1227aa275af6 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:59.230555 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 63/129] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 08:22:58 2022] PUT /identity/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394/users/21c9b4f481e04483a2d7cd25eb4e9bab/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:22:59.231693 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e633af-c063-46b8-aac7-1227aa275af6 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:59.283716 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5951a-4732-45ca-a1e5-d1c7acc5f9f9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:59.369936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb830b3-3ba0-4df7-a799-9fcf7832f395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:59.370615 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb830b3-3ba0-4df7-a799-9fcf7832f395 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:59.370903 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb830b3-3ba0-4df7-a799-9fcf7832f395 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:59.371183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb830b3-3ba0-4df7-a799-9fcf7832f395 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:59.372844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e633af-c063-46b8-aac7-1227aa275af6 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:59.389676 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb830b3-3ba0-4df7-a799-9fcf7832f395 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:22:59.397022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-ecb830b3-3ba0-4df7-a799-9fcf7832f395 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:22:59.397022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:22:59.397022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:22:59.397022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:22:59.397022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:22:59.397022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:22:59.397022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:22:59.397022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:22:59.397022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:22:59.397022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:22:59.397022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:22:59.397022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:22:59.397022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:22:59.397022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:22:59.397022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:22:59.397022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:22:59.397995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:22:59.397995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:22:59.397995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:22:59.397995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:22:59.397995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:22:59.397995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:22:59.397995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:22:59.397995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:22:59.397995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:22:59.397995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:22:59.397995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:22:59.397995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:22:59.397995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:22:59.397995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:22:59.397995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:22:59.397995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:22:59.398766 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:22:59.398766 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:22:59.398766 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:22:59.398766 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:22:59.398766 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:22:59.398766 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:22:59.398766 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:22:59.398766 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:22:59.398766 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:22:59.398766 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:22:59.398766 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:22:59.398766 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:22:59.398766 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 67/130] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:22:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:22:59.404691 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb830b3-3ba0-4df7-a799-9fcf7832f395 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:59.419981 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:59.434337 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5aee0-5cbc-4a16-9b1e-43e4a4e4543b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:59.435015 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5aee0-5cbc-4a16-9b1e-43e4a4e4543b None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:59.435298 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5aee0-5cbc-4a16-9b1e-43e4a4e4543b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:59.435687 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5aee0-5cbc-4a16-9b1e-43e4a4e4543b None admin] PATH_INFO: `/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394/users/21c9b4f481e04483a2d7cd25eb4e9bab/roles/152f4082bb67479ebaeb08c2fa683399` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:59.438402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5aee0-5cbc-4a16-9b1e-43e4a4e4543b None admin] RBAC: Authorizing `identity:create_grant(project_id=ce0954e79d28403bb7e0c4f0b78e4394, user_id=21c9b4f481e04483a2d7cd25eb4e9bab, role_id=152f4082bb67479ebaeb08c2fa683399)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:59.440450 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5aee0-5cbc-4a16-9b1e-43e4a4e4543b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:59.519134 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb830b3-3ba0-4df7-a799-9fcf7832f395 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:59.542944 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 64/131] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 08:22:59 2022] PUT /identity/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394/users/21c9b4f481e04483a2d7cd25eb4e9bab/roles/152f4082bb67479ebaeb08c2fa683399 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:22:59.545634 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ed285-3ab4-4e7e-b548-f9e5d2c326f8 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:59.547713 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ed285-3ab4-4e7e-b548-f9e5d2c326f8 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:59.548441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ed285-3ab4-4e7e-b548-f9e5d2c326f8 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:59.552366 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 65/132] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:22:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:22:59.554461 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ed285-3ab4-4e7e-b548-f9e5d2c326f8 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:59.576694 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be6a32-d2c4-4264-a476-69de398c629e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:59.577753 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be6a32-d2c4-4264-a476-69de398c629e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:59.578710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be6a32-d2c4-4264-a476-69de398c629e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:59.578710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be6a32-d2c4-4264-a476-69de398c629e None admin] PATH_INFO: `/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:59.580861 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be6a32-d2c4-4264-a476-69de398c629e None admin] RBAC: Authorizing `identity:get_project(project_id=ce0954e79d28403bb7e0c4f0b78e4394)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:59.583764 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be6a32-d2c4-4264-a476-69de398c629e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:59.587224 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 68/133] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:22:59 2022] GET /identity/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:59.589045 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78f811-6239-4d8c-8036-ac679f2e656d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:59.589579 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78f811-6239-4d8c-8036-ac679f2e656d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:59.590031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78f811-6239-4d8c-8036-ac679f2e656d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:59.610028 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:59.620632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-bf78f811-6239-4d8c-8036-ac679f2e656d None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:22:59.622519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bf78f811-6239-4d8c-8036-ac679f2e656d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:22:59.755566 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 69/134] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:22:59.756432 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78f811-6239-4d8c-8036-ac679f2e656d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:59.786499 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ed285-3ab4-4e7e-b548-f9e5d2c326f8 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:59.790036 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:59.824659 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f3de5-75bf-4fa6-bc27-7d54f5ff9c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:59.825546 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f3de5-75bf-4fa6-bc27-7d54f5ff9c42 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:59.825986 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f3de5-75bf-4fa6-bc27-7d54f5ff9c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:59.826430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f3de5-75bf-4fa6-bc27-7d54f5ff9c42 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:59.827327 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f3de5-75bf-4fa6-bc27-7d54f5ff9c42 None admin] RBAC: Adding query filter params (name=member) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:22:59.828352 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f3de5-75bf-4fa6-bc27-7d54f5ff9c42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:59.830166 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f3de5-75bf-4fa6-bc27-7d54f5ff9c42 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:59.839836 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 66/135] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 08:22:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:59.840963 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f3de5-75bf-4fa6-bc27-7d54f5ff9c42 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:22:59.892554 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78f811-6239-4d8c-8036-ac679f2e656d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:59.927671 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:22:59.931614 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ce0ee-98be-45d1-8395-4662dbede038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:59.932517 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ce0ee-98be-45d1-8395-4662dbede038 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:59.932935 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ce0ee-98be-45d1-8395-4662dbede038 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:59.933375 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ce0ee-98be-45d1-8395-4662dbede038 None admin] PATH_INFO: `/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394/users/63f02a13b768446ca706d7594e04d3fc/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:59.935635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ce0ee-98be-45d1-8395-4662dbede038 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce0954e79d28403bb7e0c4f0b78e4394, user_id=63f02a13b768446ca706d7594e04d3fc, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:59.937481 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ce0ee-98be-45d1-8395-4662dbede038 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:59.952525 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f3de5-75bf-4fa6-bc27-7d54f5ff9c42 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:22:59.989615 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ef27b-617d-48bf-9caa-90ca0ece57cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:22:59.990253 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ef27b-617d-48bf-9caa-90ca0ece57cd None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:22:59.990579 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ef27b-617d-48bf-9caa-90ca0ece57cd None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:22:59.990855 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ef27b-617d-48bf-9caa-90ca0ece57cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:22:59.991935 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ef27b-617d-48bf-9caa-90ca0ece57cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:22:59.993518 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ef27b-617d-48bf-9caa-90ca0ece57cd None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:22:59.995359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 67/136] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:22:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:22:59.995996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ef27b-617d-48bf-9caa-90ca0ece57cd None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:00.020102 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 70/137] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 08:22:59 2022] PUT /identity/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394/users/63f02a13b768446ca706d7594e04d3fc/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:23:00.021400 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ce0ee-98be-45d1-8395-4662dbede038 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:00.103022 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ef27b-617d-48bf-9caa-90ca0ece57cd None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:00.125184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:00.140355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef33a24e-de5f-4207-8472-dcf5e38dddaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:00.140997 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33a24e-de5f-4207-8472-dcf5e38dddaa None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:00.141265 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33a24e-de5f-4207-8472-dcf5e38dddaa None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:00.141544 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33a24e-de5f-4207-8472-dcf5e38dddaa None admin] PATH_INFO: `/v3/users/21c9b4f481e04483a2d7cd25eb4e9bab` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:00.142767 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef33a24e-de5f-4207-8472-dcf5e38dddaa None admin] RBAC: Authorizing `identity:get_user(user_id=21c9b4f481e04483a2d7cd25eb4e9bab)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:00.144159 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef33a24e-de5f-4207-8472-dcf5e38dddaa None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:00.146149 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 68/138] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:22:59 2022] GET /identity/v3/users/21c9b4f481e04483a2d7cd25eb4e9bab => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:00.151624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef33a24e-de5f-4207-8472-dcf5e38dddaa None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:00.175455 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ce0ee-98be-45d1-8395-4662dbede038 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:00.193529 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:00.247947 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d8f29-3224-49cf-b7db-740f3ac2a174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:00.248875 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d8f29-3224-49cf-b7db-740f3ac2a174 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:00.249330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d8f29-3224-49cf-b7db-740f3ac2a174 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:00.250482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d8f29-3224-49cf-b7db-740f3ac2a174 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:00.251406 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626d8f29-3224-49cf-b7db-740f3ac2a174 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:00.253112 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626d8f29-3224-49cf-b7db-740f3ac2a174 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:00.324347 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef33a24e-de5f-4207-8472-dcf5e38dddaa None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:00.357613 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 71/139] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 08:23:00 2022] POST /identity/v3/users => generated 283 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:23:00.368252 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1e144-912a-40f1-a226-af94c0d5fb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:00.368930 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1e144-912a-40f1-a226-af94c0d5fb87 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:00.369334 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1e144-912a-40f1-a226-af94c0d5fb87 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:00.369711 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1e144-912a-40f1-a226-af94c0d5fb87 None admin] PATH_INFO: `/v3/projects/16dec4cc338146a9bae9c5887b9b3197` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:00.370962 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa1e144-912a-40f1-a226-af94c0d5fb87 None admin] RBAC: Authorizing `identity:get_project(project_id=16dec4cc338146a9bae9c5887b9b3197)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:00.372435 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa1e144-912a-40f1-a226-af94c0d5fb87 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:00.374152 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 69/140] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:00 2022] GET /identity/v3/projects/16dec4cc338146a9bae9c5887b9b3197 => generated 312 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:00.380451 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d8f29-3224-49cf-b7db-740f3ac2a174 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:00.514472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d8f29-3224-49cf-b7db-740f3ac2a174 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:00.551201 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:00.560051 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314236e-f89c-4db7-a73b-fec0f3c6bd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:00.560737 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314236e-f89c-4db7-a73b-fec0f3c6bd84 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:00.561018 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314236e-f89c-4db7-a73b-fec0f3c6bd84 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:00.561318 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314236e-f89c-4db7-a73b-fec0f3c6bd84 None admin] PATH_INFO: `/v3/projects/16dec4cc338146a9bae9c5887b9b3197/users/21c9b4f481e04483a2d7cd25eb4e9bab/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:00.564728 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f314236e-f89c-4db7-a73b-fec0f3c6bd84 None admin] RBAC: Authorizing `identity:create_grant(project_id=16dec4cc338146a9bae9c5887b9b3197, user_id=21c9b4f481e04483a2d7cd25eb4e9bab, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:00.566283 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f314236e-f89c-4db7-a73b-fec0f3c6bd84 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:00.647162 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 72/141] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 08:23:00 2022] PUT /identity/v3/projects/16dec4cc338146a9bae9c5887b9b3197/users/21c9b4f481e04483a2d7cd25eb4e9bab/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:23:01.020456 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6b4ff-fdef-486d-8def-7f7baf544e4f None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:01.020824 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6b4ff-fdef-486d-8def-7f7baf544e4f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:01.021020 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6b4ff-fdef-486d-8def-7f7baf544e4f None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:01.022595 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 70/142] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:01.027984 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c2da1-46a8-4a0f-80b4-4f6bfb072ce7 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:01.028268 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c2da1-46a8-4a0f-80b4-4f6bfb072ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:01.028544 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c2da1-46a8-4a0f-80b4-4f6bfb072ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:01.054194 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8a8c2da1-46a8-4a0f-80b4-4f6bfb072ce7 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:01.055381 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8a8c2da1-46a8-4a0f-80b4-4f6bfb072ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:01.153585 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 73/143] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:23:01.188604 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6b4ff-fdef-486d-8def-7f7baf544e4f None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:01.297615 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:01.322792 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6b4ff-fdef-486d-8def-7f7baf544e4f None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:01.363712 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e9bb1-3584-4658-891d-80d92684568a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:01.364358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e9bb1-3584-4658-891d-80d92684568a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:01.364631 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e9bb1-3584-4658-891d-80d92684568a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:01.364897 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e9bb1-3584-4658-891d-80d92684568a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:01.379685 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e9bb1-3584-4658-891d-80d92684568a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:01.384397 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-2e6e9bb1-3584-4658-891d-80d92684568a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 08:23:01.384397 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:01.384397 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:01.384397 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:01.384397 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:01.384397 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:01.384397 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:01.384397 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:01.384397 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:01.384397 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:01.384397 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:01.384397 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:01.384397 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:01.384397 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:01.384397 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:01.384397 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:01.385254 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:01.385254 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:01.385254 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:01.385254 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:01.385254 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:01.385254 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:01.385254 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:01.385254 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:01.385254 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:01.385254 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:01.385254 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:01.385254 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:01.385254 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:01.385254 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:01.385254 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:01.385254 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:01.386043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:01.386043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:01.386043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:01.386043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:01.386043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:01.386043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:01.386043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:01.386043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:01.386043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:01.386043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:01.386043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 08:23:01.386043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:01.386043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 71/144] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:01.392352 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c2da1-46a8-4a0f-80b4-4f6bfb072ce7 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:01.518548 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c2da1-46a8-4a0f-80b4-4f6bfb072ce7 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:01.548413 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:01.566485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54b3e05-3153-4272-8f43-94713c325b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:01.567104 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b3e05-3153-4272-8f43-94713c325b2e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:01.567378 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b3e05-3153-4272-8f43-94713c325b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:01.567647 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b3e05-3153-4272-8f43-94713c325b2e None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:01.568321 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54b3e05-3153-4272-8f43-94713c325b2e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:01.568653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54b3e05-3153-4272-8f43-94713c325b2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:01.570021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54b3e05-3153-4272-8f43-94713c325b2e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:01.585311 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 74/145] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 08:23:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:01.590846 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e9bb1-3584-4658-891d-80d92684568a None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:01.700900 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e9bb1-3584-4658-891d-80d92684568a None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:01.740636 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882c3cb-54cd-416a-954d-edf6987abac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:01.741274 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882c3cb-54cd-416a-954d-edf6987abac8 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:01.741557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882c3cb-54cd-416a-954d-edf6987abac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:01.741832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882c3cb-54cd-416a-954d-edf6987abac8 None admin] PATH_INFO: `/v3/users/becd3f0974834999b872e1cbbe9a4d6b` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:01.742990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8882c3cb-54cd-416a-954d-edf6987abac8 None admin] RBAC: Authorizing `identity:get_user(user_id=becd3f0974834999b872e1cbbe9a4d6b)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:01.744442 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8882c3cb-54cd-416a-954d-edf6987abac8 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:01.746223 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 72/146] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:23:01 2022] GET /identity/v3/users/becd3f0974834999b872e1cbbe9a4d6b => generated 289 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:01.752103 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54b3e05-3153-4272-8f43-94713c325b2e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:01.869625 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54b3e05-3153-4272-8f43-94713c325b2e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:01.912977 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5538a1-44d8-4b6c-8fcb-a540fa6c6ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:01.913649 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5538a1-44d8-4b6c-8fcb-a540fa6c6ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:01.913932 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5538a1-44d8-4b6c-8fcb-a540fa6c6ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:01.914207 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5538a1-44d8-4b6c-8fcb-a540fa6c6ba8 None admin] PATH_INFO: `/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:01.915353 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5538a1-44d8-4b6c-8fcb-a540fa6c6ba8 None admin] RBAC: Authorizing `identity:get_project(project_id=ce0954e79d28403bb7e0c4f0b78e4394)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:01.916799 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5538a1-44d8-4b6c-8fcb-a540fa6c6ba8 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:01.918455 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 75/147] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:01 2022] GET /identity/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:01.925008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882c3cb-54cd-416a-954d-edf6987abac8 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:01.937649 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:02.038628 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882c3cb-54cd-416a-954d-edf6987abac8 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:02.081443 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e250cdf-be67-4450-be9b-bcae7c7f8070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:02.082133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e250cdf-be67-4450-be9b-bcae7c7f8070 None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:02.082444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e250cdf-be67-4450-be9b-bcae7c7f8070 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:02.082682 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e250cdf-be67-4450-be9b-bcae7c7f8070 None admin] PATH_INFO: `/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394/users/becd3f0974834999b872e1cbbe9a4d6b/roles/ab682c3121c04a5fb600dc47750d013c` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:02.084627 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e250cdf-be67-4450-be9b-bcae7c7f8070 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce0954e79d28403bb7e0c4f0b78e4394, user_id=becd3f0974834999b872e1cbbe9a4d6b, role_id=ab682c3121c04a5fb600dc47750d013c)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:02.086100 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e250cdf-be67-4450-be9b-bcae7c7f8070 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:02.152885 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 73/148] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 08:23:01 2022] PUT /identity/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394/users/becd3f0974834999b872e1cbbe9a4d6b/roles/ab682c3121c04a5fb600dc47750d013c => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:23:02.292696 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e5596-0ee7-4725-99d3-cd9108a12a9b None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:02.293270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e5596-0ee7-4725-99d3-cd9108a12a9b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:02.293270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e5596-0ee7-4725-99d3-cd9108a12a9b None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:02.294746 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 76/149] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:02.303738 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb97d08-6db5-4540-92c4-a4d74eec6c94 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:02.303738 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb97d08-6db5-4540-92c4-a4d74eec6c94 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:02.303738 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb97d08-6db5-4540-92c4-a4d74eec6c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:02.321685 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-4eb97d08-6db5-4540-92c4-a4d74eec6c94 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:02.323275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-4eb97d08-6db5-4540-92c4-a4d74eec6c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:02.403003 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:02.409086 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 74/150] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:02.447721 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e5596-0ee7-4725-99d3-cd9108a12a9b None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:02.520048 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:02.574983 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e5596-0ee7-4725-99d3-cd9108a12a9b None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:02.615465 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f20e54-f0c6-47e9-ab0c-88cee7efa951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:02.616108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f20e54-f0c6-47e9-ab0c-88cee7efa951 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:02.616383 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f20e54-f0c6-47e9-ab0c-88cee7efa951 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:02.616646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f20e54-f0c6-47e9-ab0c-88cee7efa951 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:02.627991 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f20e54-f0c6-47e9-ab0c-88cee7efa951 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:02.633363 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-60f20e54-f0c6-47e9-ab0c-88cee7efa951 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:23:02.633363 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:02.633363 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:02.633363 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:02.633363 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:02.633363 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:02.633363 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:02.633363 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:02.633363 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:02.633363 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:02.633363 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:02.633363 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:02.633363 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:02.633363 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:02.633363 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:02.633363 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:02.634210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:02.634210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:02.634210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:02.634210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:02.634210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:02.634210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:02.634210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:02.634210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:02.634210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:02.634210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:02.634210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:02.634210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:02.634210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:02.634210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:02.634210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:02.634210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:02.635358 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:02.635358 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:02.635358 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:02.635358 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:02.635358 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:02.635358 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:02.635358 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:02.635358 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:02.635358 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:02.635358 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:02.635358 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:23:02.635358 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:02.635358 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 77/151] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:02.641183 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb97d08-6db5-4540-92c4-a4d74eec6c94 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:02.649657 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335eba8-9f45-468f-b7e9-d0322e8debf7 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:02.651707 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335eba8-9f45-468f-b7e9-d0322e8debf7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:02.651989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335eba8-9f45-468f-b7e9-d0322e8debf7 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:02.654674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 78/152] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:02 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:02.663604 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881844a-d5c3-43e0-aa31-d1e3f0f810e7 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:02.663881 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881844a-d5c3-43e0-aa31-d1e3f0f810e7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:02.664200 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881844a-d5c3-43e0-aa31-d1e3f0f810e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:02.692577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-2881844a-d5c3-43e0-aa31-d1e3f0f810e7 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:02.693782 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2881844a-d5c3-43e0-aa31-d1e3f0f810e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:02.831371 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb97d08-6db5-4540-92c4-a4d74eec6c94 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:02.858552 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 79/153] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 08 08:23:02.896991 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881844a-d5c3-43e0-aa31-d1e3f0f810e7 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:02.911525 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22077f-7720-4b7f-b17c-13ccce0f1fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:02.911525 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22077f-7720-4b7f-b17c-13ccce0f1fbc None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:02.911525 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22077f-7720-4b7f-b17c-13ccce0f1fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:02.911525 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22077f-7720-4b7f-b17c-13ccce0f1fbc None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:02.911885 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22077f-7720-4b7f-b17c-13ccce0f1fbc None admin] RBAC: Adding query filter params (name=member) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:02.911885 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22077f-7720-4b7f-b17c-13ccce0f1fbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:02.911885 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22077f-7720-4b7f-b17c-13ccce0f1fbc None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:02.923946 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 75/154] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 08:23:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:02.929687 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22077f-7720-4b7f-b17c-13ccce0f1fbc None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:02.980613 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:03.017469 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:03.096342 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22077f-7720-4b7f-b17c-13ccce0f1fbc None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:03.125490 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881844a-d5c3-43e0-aa31-d1e3f0f810e7 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:03.149037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c9952-a452-464d-a294-98d90749d2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:03.149752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c9952-a452-464d-a294-98d90749d2f7 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:03.150031 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c9952-a452-464d-a294-98d90749d2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:03.150297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c9952-a452-464d-a294-98d90749d2f7 None admin] PATH_INFO: `/v3/users/21c9b4f481e04483a2d7cd25eb4e9bab` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:03.153822 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c9952-a452-464d-a294-98d90749d2f7 None admin] RBAC: Authorizing `identity:get_user(user_id=21c9b4f481e04483a2d7cd25eb4e9bab)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:03.155300 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c9952-a452-464d-a294-98d90749d2f7 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:03.157633 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 76/155] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:23:02 2022] GET /identity/v3/users/21c9b4f481e04483a2d7cd25eb4e9bab => generated 275 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:03.164076 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c9952-a452-464d-a294-98d90749d2f7 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:03.209999 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f7fc86-e478-45e9-8135-24f8f5165f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:03.210775 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7fc86-e478-45e9-8135-24f8f5165f9d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:03.211059 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7fc86-e478-45e9-8135-24f8f5165f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:03.211328 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7fc86-e478-45e9-8135-24f8f5165f9d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:03.234200 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f7fc86-e478-45e9-8135-24f8f5165f9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:03.239238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-d1f7fc86-e478-45e9-8135-24f8f5165f9d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 08:23:03.239238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:03.239238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:03.239238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:03.239238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:03.239238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:03.239238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:03.239238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:03.239238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:03.239238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:03.239238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:03.239238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:03.239238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:03.239238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:03.239238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:03.239238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:03.240174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:03.240174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:03.240174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:03.240174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:03.240174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:03.240174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:03.240174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:03.240174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:03.240174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:03.240174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:03.240174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:03.240174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:03.240174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:03.240174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:03.240174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:03.240174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:03.241385 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:03.241385 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:03.241385 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:03.241385 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:03.241385 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:03.241385 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:03.241385 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:03.241385 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:03.241385 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:03.241385 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:03.241385 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 08:23:03.241385 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:03.241385 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 80/156] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 08:23:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:23:03.246593 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f7fc86-e478-45e9-8135-24f8f5165f9d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:03.291797 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c9952-a452-464d-a294-98d90749d2f7 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:03.321534 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:03.327293 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f4b81-f30b-4b97-a799-d93234cd48fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:03.328141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f4b81-f30b-4b97-a799-d93234cd48fc None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:03.328367 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f4b81-f30b-4b97-a799-d93234cd48fc None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:03.328653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f4b81-f30b-4b97-a799-d93234cd48fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:03.398992 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f4b81-f30b-4b97-a799-d93234cd48fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:03.400541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f4b81-f30b-4b97-a799-d93234cd48fc None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:03.449524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:03.454104 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f7fc86-e478-45e9-8135-24f8f5165f9d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:03.505531 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558252b8-386e-4409-994b-ad00d1ce841b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:03.506149 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558252b8-386e-4409-994b-ad00d1ce841b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:03.506425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558252b8-386e-4409-994b-ad00d1ce841b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:03.506799 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558252b8-386e-4409-994b-ad00d1ce841b None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:03.507712 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 77/157] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:23:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:23:03.508135 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558252b8-386e-4409-994b-ad00d1ce841b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:03.508135 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558252b8-386e-4409-994b-ad00d1ce841b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:03.509554 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558252b8-386e-4409-994b-ad00d1ce841b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:03.513433 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f4b81-f30b-4b97-a799-d93234cd48fc None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:03.523108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 81/158] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:23:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:03.528782 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558252b8-386e-4409-994b-ad00d1ce841b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:03.542193 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:03.618634 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f4b81-f30b-4b97-a799-d93234cd48fc None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:03.656761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a3602c-a665-43ec-b3dc-e55fe7ece395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:03.657454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3602c-a665-43ec-b3dc-e55fe7ece395 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:03.657782 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3602c-a665-43ec-b3dc-e55fe7ece395 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:03.658437 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3602c-a665-43ec-b3dc-e55fe7ece395 None admin] PATH_INFO: `/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:03.659665 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a3602c-a665-43ec-b3dc-e55fe7ece395 None admin] RBAC: Authorizing `identity:get_project(project_id=ce0954e79d28403bb7e0c4f0b78e4394)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:03.661101 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a3602c-a665-43ec-b3dc-e55fe7ece395 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:03.663217 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 78/159] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:03 2022] GET /identity/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:03.669712 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a3602c-a665-43ec-b3dc-e55fe7ece395 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:03.753303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558252b8-386e-4409-994b-ad00d1ce841b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:03.809523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:03.815072 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a3602c-a665-43ec-b3dc-e55fe7ece395 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:03.821861 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38cefd6-e71a-4226-8aaf-8eab54073a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:03.839192 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38cefd6-e71a-4226-8aaf-8eab54073a73 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:03.839192 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38cefd6-e71a-4226-8aaf-8eab54073a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:03.839192 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38cefd6-e71a-4226-8aaf-8eab54073a73 None admin] PATH_INFO: `/v3/users/21c9b4f481e04483a2d7cd25eb4e9bab` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:03.839192 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38cefd6-e71a-4226-8aaf-8eab54073a73 None admin] RBAC: Authorizing `identity:get_user(user_id=21c9b4f481e04483a2d7cd25eb4e9bab)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:03.839192 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38cefd6-e71a-4226-8aaf-8eab54073a73 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:03.839192 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 82/160] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:23:03 2022] GET /identity/v3/users/21c9b4f481e04483a2d7cd25eb4e9bab => generated 275 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:03.839615 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382e0fd-9736-4b4d-bb8b-39764e6d6930 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:03.839615 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382e0fd-9736-4b4d-bb8b-39764e6d6930 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:03.839615 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382e0fd-9736-4b4d-bb8b-39764e6d6930 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:03.839615 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 83/161] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:03.842441 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382e0fd-9736-4b4d-bb8b-39764e6d6930 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:03.865407 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c53d3-31d1-4b7f-bdb9-005e1fb9b89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:03.866035 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c53d3-31d1-4b7f-bdb9-005e1fb9b89c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:03.866309 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c53d3-31d1-4b7f-bdb9-005e1fb9b89c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:03.866576 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c53d3-31d1-4b7f-bdb9-005e1fb9b89c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:03.867427 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c53d3-31d1-4b7f-bdb9-005e1fb9b89c None admin] RBAC: Adding query filter params (role.id=b0ba6e2bf793402abe2e680c2a7dcb53, scope.project.id=ce0954e79d28403bb7e0c4f0b78e4394, user.id=21c9b4f481e04483a2d7cd25eb4e9bab) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:03.867740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c53d3-31d1-4b7f-bdb9-005e1fb9b89c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:03.869198 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c53d3-31d1-4b7f-bdb9-005e1fb9b89c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:03.875989 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 79/162] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:23:03 2022] GET /identity/v3/role_assignments?user.id=21c9b4f481e04483a2d7cd25eb4e9bab&scope.project.id=ce0954e79d28403bb7e0c4f0b78e4394&role.id=b0ba6e2bf793402abe2e680c2a7dcb53 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:03.881439 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b77c11-d1ed-47d2-bf40-615ddfe61ad0 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:03.881750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b77c11-d1ed-47d2-bf40-615ddfe61ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:03.882068 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b77c11-d1ed-47d2-bf40-615ddfe61ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:03.897554 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:03.908659 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-b5b77c11-d1ed-47d2-bf40-615ddfe61ad0 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:03.910014 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b5b77c11-d1ed-47d2-bf40-615ddfe61ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:04.011104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 80/163] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:04.027171 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382e0fd-9736-4b4d-bb8b-39764e6d6930 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:04.055793 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b77c11-d1ed-47d2-bf40-615ddfe61ad0 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:04.085953 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644ce54-18d5-4acc-9171-7059cff8c67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:04.086592 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644ce54-18d5-4acc-9171-7059cff8c67d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:04.086870 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644ce54-18d5-4acc-9171-7059cff8c67d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:04.087141 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644ce54-18d5-4acc-9171-7059cff8c67d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:04.203914 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644ce54-18d5-4acc-9171-7059cff8c67d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:04.205712 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644ce54-18d5-4acc-9171-7059cff8c67d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:04.209247 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:04.223482 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b77c11-d1ed-47d2-bf40-615ddfe61ad0 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:04.252208 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:04.279592 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a7cb61-1334-4a48-b871-d0a2e7dbcc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:04.280581 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7cb61-1334-4a48-b871-d0a2e7dbcc43 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:04.281372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7cb61-1334-4a48-b871-d0a2e7dbcc43 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:04.282083 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7cb61-1334-4a48-b871-d0a2e7dbcc43 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:04.306186 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a7cb61-1334-4a48-b871-d0a2e7dbcc43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:04.311761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-54a7cb61-1334-4a48-b871-d0a2e7dbcc43 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:23:04.311761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:04.311761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:04.311761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:04.311761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:04.311761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:04.311761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:04.311761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:04.311761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:04.311761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:04.311761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:04.311761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:04.311761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:04.311761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:04.311761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:04.311761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:04.312750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:04.312750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:04.312750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:04.312750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:04.312750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:04.312750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:04.312750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:04.312750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:04.312750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:04.312750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:04.312750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:04.312750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:04.312750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:04.312750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:04.312750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:04.312750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:04.318024 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:04.318024 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:04.318024 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:04.318024 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:04.318024 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:04.318024 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:04.318024 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:04.318024 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:04.318024 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:04.318024 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:04.318024 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:23:04.318024 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:04.320522 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 81/164] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:23:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:04.326726 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a7cb61-1334-4a48-b871-d0a2e7dbcc43 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:04.359726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 84/165] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:23:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:23:04.365099 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644ce54-18d5-4acc-9171-7059cff8c67d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:04.497665 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a7cb61-1334-4a48-b871-d0a2e7dbcc43 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:04.516275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644ce54-18d5-4acc-9171-7059cff8c67d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:04.549000 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:04.554889 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b0a113-17e5-4a6f-81d4-50ca93d3789e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:04.556084 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b0a113-17e5-4a6f-81d4-50ca93d3789e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:04.556733 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b0a113-17e5-4a6f-81d4-50ca93d3789e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:04.557229 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b0a113-17e5-4a6f-81d4-50ca93d3789e None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:04.558121 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b0a113-17e5-4a6f-81d4-50ca93d3789e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:04.558587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b0a113-17e5-4a6f-81d4-50ca93d3789e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:04.560457 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b0a113-17e5-4a6f-81d4-50ca93d3789e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:04.565941 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e56a3-3b7a-4df6-bdcd-2af342eaa3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:04.566599 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e56a3-3b7a-4df6-bdcd-2af342eaa3db None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:04.566883 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e56a3-3b7a-4df6-bdcd-2af342eaa3db None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:04.567157 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e56a3-3b7a-4df6-bdcd-2af342eaa3db None admin] PATH_INFO: `/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:04.568362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e56a3-3b7a-4df6-bdcd-2af342eaa3db None admin] RBAC: Authorizing `identity:get_project(project_id=ce0954e79d28403bb7e0c4f0b78e4394)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:04.569882 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e56a3-3b7a-4df6-bdcd-2af342eaa3db None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:04.571848 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 85/166] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:04 2022] GET /identity/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:04.575366 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 82/167] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:04.577122 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e56a3-3b7a-4df6-bdcd-2af342eaa3db None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:04.583117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b0a113-17e5-4a6f-81d4-50ca93d3789e None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:04.608732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:04.704435 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e56a3-3b7a-4df6-bdcd-2af342eaa3db None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:04.742121 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fbb69f-69e9-4934-8d36-6b2497a2721c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:04.742789 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbb69f-69e9-4934-8d36-6b2497a2721c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:04.743074 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbb69f-69e9-4934-8d36-6b2497a2721c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:04.743347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbb69f-69e9-4934-8d36-6b2497a2721c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:04.744088 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fbb69f-69e9-4934-8d36-6b2497a2721c None admin] RBAC: Adding query filter params (role.id=152f4082bb67479ebaeb08c2fa683399, scope.project.id=ce0954e79d28403bb7e0c4f0b78e4394, user.id=21c9b4f481e04483a2d7cd25eb4e9bab) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:04.744394 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fbb69f-69e9-4934-8d36-6b2497a2721c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:04.745871 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fbb69f-69e9-4934-8d36-6b2497a2721c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:04.751809 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 86/168] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:23:04 2022] GET /identity/v3/role_assignments?user.id=21c9b4f481e04483a2d7cd25eb4e9bab&scope.project.id=ce0954e79d28403bb7e0c4f0b78e4394&role.id=152f4082bb67479ebaeb08c2fa683399 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:04.753005 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbc61f-85c4-4556-8013-d95c274d9794 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:04.755147 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbc61f-85c4-4556-8013-d95c274d9794 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:04.755438 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbc61f-85c4-4556-8013-d95c274d9794 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:04.756606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b0a113-17e5-4a6f-81d4-50ca93d3789e None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:04.756796 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 87/169] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:04.762425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31315f03-f41d-4edf-93d8-3fc450a2dc6f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:04.762724 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31315f03-f41d-4edf-93d8-3fc450a2dc6f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:04.762976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31315f03-f41d-4edf-93d8-3fc450a2dc6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:04.782129 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-31315f03-f41d-4edf-93d8-3fc450a2dc6f None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:04.783338 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-31315f03-f41d-4edf-93d8-3fc450a2dc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:04.818388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eec2df-cc35-40ca-85cd-058b9a22f85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:04.819411 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eec2df-cc35-40ca-85cd-058b9a22f85c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:04.819910 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eec2df-cc35-40ca-85cd-058b9a22f85c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:04.820379 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eec2df-cc35-40ca-85cd-058b9a22f85c None admin] PATH_INFO: `/v3/users/63f02a13b768446ca706d7594e04d3fc` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:04.825861 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eec2df-cc35-40ca-85cd-058b9a22f85c None admin] RBAC: Authorizing `identity:get_user(user_id=63f02a13b768446ca706d7594e04d3fc)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:04.827790 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eec2df-cc35-40ca-85cd-058b9a22f85c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:04.830211 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 83/170] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:23:04 2022] GET /identity/v3/users/63f02a13b768446ca706d7594e04d3fc => generated 247 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:04.836500 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eec2df-cc35-40ca-85cd-058b9a22f85c None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:04.844686 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:04.855780 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 88/171] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:23:04.881453 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31315f03-f41d-4edf-93d8-3fc450a2dc6f None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:04.989393 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:05.009068 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eec2df-cc35-40ca-85cd-058b9a22f85c None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:05.019918 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31315f03-f41d-4edf-93d8-3fc450a2dc6f None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:05.045689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe36be-bf3b-4434-b352-272fd08aff38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:05.046318 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe36be-bf3b-4434-b352-272fd08aff38 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:05.047326 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe36be-bf3b-4434-b352-272fd08aff38 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:05.047714 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe36be-bf3b-4434-b352-272fd08aff38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:05.082057 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347615ec-a8ed-4994-a263-379f94e07834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:05.082651 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347615ec-a8ed-4994-a263-379f94e07834 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:05.082939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347615ec-a8ed-4994-a263-379f94e07834 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:05.083212 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347615ec-a8ed-4994-a263-379f94e07834 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:05.107241 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347615ec-a8ed-4994-a263-379f94e07834 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:05.120596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-347615ec-a8ed-4994-a263-379f94e07834 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:23:05.120596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:05.120596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:05.120596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:05.120596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:05.120596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:05.120596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:05.120596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:05.120596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:05.120596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:05.120596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:05.120596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:05.120596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:05.120596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:05.120596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:05.120596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:05.121639 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:05.121639 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:05.121639 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:05.121639 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:05.121639 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:05.121639 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:05.121639 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:05.121639 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:05.121639 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:05.121639 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:05.121639 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:05.121639 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:05.121639 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:05.121639 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:05.121639 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:05.121639 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:05.122862 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:05.122862 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:05.122862 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:05.122862 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:05.122862 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:05.122862 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:05.122862 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:05.122862 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:05.122862 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:05.122862 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:05.122862 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:23:05.122862 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:05.122862 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 89/172] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:23:05.128506 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347615ec-a8ed-4994-a263-379f94e07834 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:05.142546 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe36be-bf3b-4434-b352-272fd08aff38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:05.144030 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe36be-bf3b-4434-b352-272fd08aff38 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:05.237536 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:05.256852 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:05.299628 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 84/173] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:23:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 08:23:05.305795 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe36be-bf3b-4434-b352-272fd08aff38 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:05.315891 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347615ec-a8ed-4994-a263-379f94e07834 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:05.369869 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e044f-a123-46ae-9f52-8e6378a12f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:05.370510 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e044f-a123-46ae-9f52-8e6378a12f60 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:05.370795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e044f-a123-46ae-9f52-8e6378a12f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:05.371071 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e044f-a123-46ae-9f52-8e6378a12f60 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:05.371757 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e044f-a123-46ae-9f52-8e6378a12f60 None admin] RBAC: Adding query filter params (name=member) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:05.372061 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e044f-a123-46ae-9f52-8e6378a12f60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:05.373477 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e044f-a123-46ae-9f52-8e6378a12f60 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:05.383192 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 90/174] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 08:23:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:05.390827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e044f-a123-46ae-9f52-8e6378a12f60 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:05.447634 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe36be-bf3b-4434-b352-272fd08aff38 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:05.489741 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d38fd3-97f5-4758-8f7f-51dd7b732eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:05.490393 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d38fd3-97f5-4758-8f7f-51dd7b732eda None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:05.491163 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d38fd3-97f5-4758-8f7f-51dd7b732eda None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:05.491372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d38fd3-97f5-4758-8f7f-51dd7b732eda None admin] PATH_INFO: `/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:05.492808 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d38fd3-97f5-4758-8f7f-51dd7b732eda None admin] RBAC: Authorizing `identity:get_project(project_id=ce0954e79d28403bb7e0c4f0b78e4394)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:05.494292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d38fd3-97f5-4758-8f7f-51dd7b732eda None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:05.495958 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 85/175] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:05 2022] GET /identity/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:05.503754 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d38fd3-97f5-4758-8f7f-51dd7b732eda None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:05.569508 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:05.612257 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d38fd3-97f5-4758-8f7f-51dd7b732eda None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:05.618307 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e044f-a123-46ae-9f52-8e6378a12f60 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:05.636231 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:05.649552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbaa64e-0f8e-4ccf-9682-d221a30b91a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:05.650220 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaa64e-0f8e-4ccf-9682-d221a30b91a2 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:05.651366 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaa64e-0f8e-4ccf-9682-d221a30b91a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:05.651366 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaa64e-0f8e-4ccf-9682-d221a30b91a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:05.651530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbaa64e-0f8e-4ccf-9682-d221a30b91a2 None admin] RBAC: Adding query filter params (role.id=c1e514a4b5424f959b069f36cbfd8147, scope.project.id=ce0954e79d28403bb7e0c4f0b78e4394, user.id=63f02a13b768446ca706d7594e04d3fc) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:05.651826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbaa64e-0f8e-4ccf-9682-d221a30b91a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:05.653301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbaa64e-0f8e-4ccf-9682-d221a30b91a2 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:05.659775 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 86/176] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:23:05 2022] GET /identity/v3/role_assignments?user.id=63f02a13b768446ca706d7594e04d3fc&scope.project.id=ce0954e79d28403bb7e0c4f0b78e4394&role.id=c1e514a4b5424f959b069f36cbfd8147 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:05.668494 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372bf176-3a5e-48a7-a08d-6a6f97f25265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:05.669171 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bf176-3a5e-48a7-a08d-6a6f97f25265 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:05.669479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bf176-3a5e-48a7-a08d-6a6f97f25265 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:05.669751 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bf176-3a5e-48a7-a08d-6a6f97f25265 None admin] PATH_INFO: `/v3/users/21c9b4f481e04483a2d7cd25eb4e9bab` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:05.670965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372bf176-3a5e-48a7-a08d-6a6f97f25265 None admin] RBAC: Authorizing `identity:get_user(user_id=21c9b4f481e04483a2d7cd25eb4e9bab)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:05.673411 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372bf176-3a5e-48a7-a08d-6a6f97f25265 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:05.677493 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 91/177] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:23:05 2022] GET /identity/v3/users/21c9b4f481e04483a2d7cd25eb4e9bab => generated 275 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:05.681346 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbaa64e-0f8e-4ccf-9682-d221a30b91a2 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:05.791934 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbaa64e-0f8e-4ccf-9682-d221a30b91a2 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:05.806873 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:05.829812 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab02667-7d53-4ac8-b04e-db231b6c7512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:05.830687 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab02667-7d53-4ac8-b04e-db231b6c7512 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:05.830968 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab02667-7d53-4ac8-b04e-db231b6c7512 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:05.831240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab02667-7d53-4ac8-b04e-db231b6c7512 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:05.834919 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62c210-6598-40ec-aec0-52655f7300cc None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:05.835200 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62c210-6598-40ec-aec0-52655f7300cc None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:05.835472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62c210-6598-40ec-aec0-52655f7300cc None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:05.837277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 92/178] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:05.844811 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eded7e-5b0e-4713-aecc-306cbff6e548 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:05.845134 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eded7e-5b0e-4713-aecc-306cbff6e548 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:05.845425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eded7e-5b0e-4713-aecc-306cbff6e548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:05.881379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-04eded7e-5b0e-4713-aecc-306cbff6e548 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:05.882569 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-04eded7e-5b0e-4713-aecc-306cbff6e548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:05.928287 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab02667-7d53-4ac8-b04e-db231b6c7512 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:05.929815 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab02667-7d53-4ac8-b04e-db231b6c7512 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:05.998721 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 93/179] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:06.001003 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21733f1-acc4-4fc2-80a7-d7be87787d70 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:06.001418 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21733f1-acc4-4fc2-80a7-d7be87787d70 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:06.001700 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21733f1-acc4-4fc2-80a7-d7be87787d70 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:06.003197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 94/180] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:06.009360 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb1023-4d2b-4450-b0f5-598f3ec815dd None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:06.009646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb1023-4d2b-4450-b0f5-598f3ec815dd None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:06.009918 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb1023-4d2b-4450-b0f5-598f3ec815dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:06.032202 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-92cb1023-4d2b-4450-b0f5-598f3ec815dd None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:06.033550 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-92cb1023-4d2b-4450-b0f5-598f3ec815dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:06.060128 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 87/181] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:23:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:23:06.060893 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab02667-7d53-4ac8-b04e-db231b6c7512 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:06.081219 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:06.121313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 95/182] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:23:06.123048 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb1023-4d2b-4450-b0f5-598f3ec815dd None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:06.177632 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab02667-7d53-4ac8-b04e-db231b6c7512 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:06.191389 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:06.215908 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3f2d3-8b3e-4da7-a993-111ff310daa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:06.216823 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3f2d3-8b3e-4da7-a993-111ff310daa3 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:06.217272 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3f2d3-8b3e-4da7-a993-111ff310daa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:06.217680 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3f2d3-8b3e-4da7-a993-111ff310daa3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:06.229628 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef3f2d3-8b3e-4da7-a993-111ff310daa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:06.235359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-1ef3f2d3-8b3e-4da7-a993-111ff310daa3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:23:06.235359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:06.235359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:06.235359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:06.235359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:06.235359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:06.235359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:06.235359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:06.235359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:06.235359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:06.235359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:06.235359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:06.235359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:06.235359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:06.235359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:06.235359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:06.236441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:06.236441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:06.236441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:06.236441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:06.236441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:06.236441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:06.236441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:06.236441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:06.236441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:06.236441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:06.236441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:06.236441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:06.236441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:06.236441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:06.236441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:06.236441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:06.237466 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:06.237466 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:06.237466 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:06.237466 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:06.237466 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:06.237466 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:06.237466 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:06.237466 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:06.237466 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:06.237466 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:06.237466 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:23:06.237466 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:06.239489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 88/183] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:23:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:06.240485 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3f2d3-8b3e-4da7-a993-111ff310daa3 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:06.269473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb1023-4d2b-4450-b0f5-598f3ec815dd None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:06.310401 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b929815-9314-4383-a098-dcfc8c25d33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:06.311055 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b929815-9314-4383-a098-dcfc8c25d33f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:06.311337 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b929815-9314-4383-a098-dcfc8c25d33f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:06.311612 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b929815-9314-4383-a098-dcfc8c25d33f None admin] PATH_INFO: `/v3/projects/16dec4cc338146a9bae9c5887b9b3197` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:06.313786 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b929815-9314-4383-a098-dcfc8c25d33f None admin] RBAC: Authorizing `identity:get_project(project_id=16dec4cc338146a9bae9c5887b9b3197)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:06.315245 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b929815-9314-4383-a098-dcfc8c25d33f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:06.316888 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 96/184] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:06 2022] GET /identity/v3/projects/16dec4cc338146a9bae9c5887b9b3197 => generated 312 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:06.318171 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd8715-843a-4995-ab9b-d3f701e1401c None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:06.318459 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd8715-843a-4995-ab9b-d3f701e1401c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:06.318732 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd8715-843a-4995-ab9b-d3f701e1401c None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:06.320079 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 97/185] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:06.320689 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd8715-843a-4995-ab9b-d3f701e1401c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:06.336705 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:06.368629 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3f2d3-8b3e-4da7-a993-111ff310daa3 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:06.405544 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59891546-602f-44d5-9195-a23d09849573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:06.406378 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59891546-602f-44d5-9195-a23d09849573 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:06.406855 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59891546-602f-44d5-9195-a23d09849573 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:06.408382 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59891546-602f-44d5-9195-a23d09849573 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:06.410412 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59891546-602f-44d5-9195-a23d09849573 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:06.410843 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59891546-602f-44d5-9195-a23d09849573 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:06.413875 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 89/186] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:06.418053 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59891546-602f-44d5-9195-a23d09849573 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:06.434467 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd8715-843a-4995-ab9b-d3f701e1401c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:06.444144 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:06.468314 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e91a12a-076a-4be4-8954-f2da8c86d90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:06.469716 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91a12a-076a-4be4-8954-f2da8c86d90f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:06.470157 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91a12a-076a-4be4-8954-f2da8c86d90f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:06.470416 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91a12a-076a-4be4-8954-f2da8c86d90f None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:06.471091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e91a12a-076a-4be4-8954-f2da8c86d90f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:06.471394 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e91a12a-076a-4be4-8954-f2da8c86d90f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:06.472765 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e91a12a-076a-4be4-8954-f2da8c86d90f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:06.480601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 98/187] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:06.482652 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fbf40-3d15-49ef-a128-28402a1f172b None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:06.482943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fbf40-3d15-49ef-a128-28402a1f172b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:06.483217 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fbf40-3d15-49ef-a128-28402a1f172b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:06.501328 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b84fbf40-3d15-49ef-a128-28402a1f172b None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:06.502512 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b84fbf40-3d15-49ef-a128-28402a1f172b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:06.525687 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59891546-602f-44d5-9195-a23d09849573 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:06.558864 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220fe18-8a57-42e9-8e39-c54d895fd44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:06.559470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220fe18-8a57-42e9-8e39-c54d895fd44c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:06.559763 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220fe18-8a57-42e9-8e39-c54d895fd44c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:06.560033 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220fe18-8a57-42e9-8e39-c54d895fd44c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:06.560739 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b220fe18-8a57-42e9-8e39-c54d895fd44c None admin] RBAC: Adding query filter params (role.id=b0ba6e2bf793402abe2e680c2a7dcb53, scope.project.id=16dec4cc338146a9bae9c5887b9b3197, user.id=21c9b4f481e04483a2d7cd25eb4e9bab) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:06.561046 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b220fe18-8a57-42e9-8e39-c54d895fd44c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:06.562946 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b220fe18-8a57-42e9-8e39-c54d895fd44c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:06.568075 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 99/188] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:06.574620 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 90/189] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:23:06 2022] GET /identity/v3/role_assignments?user.id=21c9b4f481e04483a2d7cd25eb4e9bab&scope.project.id=16dec4cc338146a9bae9c5887b9b3197&role.id=b0ba6e2bf793402abe2e680c2a7dcb53 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:06.575123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84fbf40-3d15-49ef-a128-28402a1f172b None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:06.575202 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6a500-f002-4cb0-a4dd-b03a677e470e None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:06.575202 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6a500-f002-4cb0-a4dd-b03a677e470e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:06.575202 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6a500-f002-4cb0-a4dd-b03a677e470e None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:06.575202 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 91/190] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:06.575202 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6a500-f002-4cb0-a4dd-b03a677e470e None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:06.583861 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:06.686446 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:06.689992 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84fbf40-3d15-49ef-a128-28402a1f172b None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:06.700903 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6a500-f002-4cb0-a4dd-b03a677e470e None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:06.735234 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff158e4-1348-4386-80c6-8d2b3a3405eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:06.735866 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff158e4-1348-4386-80c6-8d2b3a3405eb None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:06.736144 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff158e4-1348-4386-80c6-8d2b3a3405eb None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:06.736414 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff158e4-1348-4386-80c6-8d2b3a3405eb None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:06.737066 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff158e4-1348-4386-80c6-8d2b3a3405eb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:06.738510 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8af0a6-e48c-4004-a1a0-a4f2eacfcab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:06.738634 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff158e4-1348-4386-80c6-8d2b3a3405eb None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:06.739177 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8af0a6-e48c-4004-a1a0-a4f2eacfcab0 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:06.739514 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8af0a6-e48c-4004-a1a0-a4f2eacfcab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:06.739854 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8af0a6-e48c-4004-a1a0-a4f2eacfcab0 None admin] PATH_INFO: `/v3/users/b369df7c929c476ebfc35f907c63d466` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:06.755238 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8af0a6-e48c-4004-a1a0-a4f2eacfcab0 None admin] RBAC: Authorizing `identity:get_user(user_id=b369df7c929c476ebfc35f907c63d466)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:06.756687 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8af0a6-e48c-4004-a1a0-a4f2eacfcab0 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:06.758416 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 92/191] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:23:06 2022] GET /identity/v3/users/b369df7c929c476ebfc35f907c63d466 => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:06.759556 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc34fd9-bb7b-4048-8575-d5c66a23e874 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:06.759834 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc34fd9-bb7b-4048-8575-d5c66a23e874 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:06.764760 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc34fd9-bb7b-4048-8575-d5c66a23e874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:06.778392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-5cc34fd9-bb7b-4048-8575-d5c66a23e874 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:06.779558 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5cc34fd9-bb7b-4048-8575-d5c66a23e874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:06.835850 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:06.846082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 93/192] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:06.846838 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc34fd9-bb7b-4048-8575-d5c66a23e874 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:06.850141 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 100/193] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 08:23:06 2022] POST /identity/v3/users => generated 297 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:23:06.862455 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff158e4-1348-4386-80c6-8d2b3a3405eb None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:06.966430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc34fd9-bb7b-4048-8575-d5c66a23e874 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:06.998129 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff158e4-1348-4386-80c6-8d2b3a3405eb None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:07.000136 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ca757-95ca-4468-b32b-3ce01afc1013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:07.000788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ca757-95ca-4468-b32b-3ce01afc1013 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:07.001057 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ca757-95ca-4468-b32b-3ce01afc1013 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:07.001370 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ca757-95ca-4468-b32b-3ce01afc1013 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:07.012257 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ca757-95ca-4468-b32b-3ce01afc1013 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:07.018130 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-c29ca757-95ca-4468-b32b-3ce01afc1013 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 08:23:07.018130 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:07.018130 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:07.018130 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:07.018130 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:07.018130 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:07.018130 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:07.018130 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:07.018130 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:07.018130 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:07.018130 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:07.018130 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:07.018130 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:07.018130 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:07.018130 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:07.018130 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:07.019006 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:07.019006 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:07.019006 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:07.019006 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:07.019006 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:07.019006 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:07.019006 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:07.019006 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:07.019006 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:07.019006 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:07.019006 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:07.019006 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:07.019006 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:07.019006 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:07.019006 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:07.019006 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:07.019806 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:07.019806 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:07.019806 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:07.019806 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:07.019806 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:07.019806 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:07.019806 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:07.019806 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:07.019806 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:07.019806 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:07.019806 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 08:23:07.019806 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:07.019806 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 94/194] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 08:23:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:07.020458 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ca757-95ca-4468-b32b-3ce01afc1013 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:07.046996 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a725c5b-e63c-4734-8700-9bacec1d8e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:07.047898 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a725c5b-e63c-4734-8700-9bacec1d8e95 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:07.048184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a725c5b-e63c-4734-8700-9bacec1d8e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:07.048463 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a725c5b-e63c-4734-8700-9bacec1d8e95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:07.081635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:07.084185 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:07.140014 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ca757-95ca-4468-b32b-3ce01afc1013 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:07.141946 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a725c5b-e63c-4734-8700-9bacec1d8e95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:07.143454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a725c5b-e63c-4734-8700-9bacec1d8e95 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:07.180040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3738606-b10a-4949-9dc9-f21cfbc5eef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:07.180772 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3738606-b10a-4949-9dc9-f21cfbc5eef6 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:07.181196 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3738606-b10a-4949-9dc9-f21cfbc5eef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:07.181543 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3738606-b10a-4949-9dc9-f21cfbc5eef6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:07.193377 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3738606-b10a-4949-9dc9-f21cfbc5eef6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:07.199641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-f3738606-b10a-4949-9dc9-f21cfbc5eef6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 08:23:07.199641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:07.199641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:07.199641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:07.199641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:07.199641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:07.199641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:07.199641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:07.199641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:07.199641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:07.199641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:07.199641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:07.199641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:07.199641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:07.199641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:07.199641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:07.200586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:07.200586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:07.200586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:07.200586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:07.200586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:07.200586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:07.200586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:07.200586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:07.200586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:07.200586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:07.200586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:07.200586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:07.200586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:07.200586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:07.200586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:07.200586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:07.201413 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:07.201413 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:07.201413 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:07.201413 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:07.201413 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:07.201413 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:07.201413 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:07.201413 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:07.201413 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:07.201413 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:07.201413 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 08:23:07.201413 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:07.202018 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 95/195] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:07.203097 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3738606-b10a-4949-9dc9-f21cfbc5eef6 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:07.286766 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 101/196] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:23:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:23:07.287808 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a725c5b-e63c-4734-8700-9bacec1d8e95 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:07.329684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3738606-b10a-4949-9dc9-f21cfbc5eef6 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:07.350003 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:07.367786 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:07.390975 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d5971-70b5-4468-ad35-45ea04233628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:07.391583 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d5971-70b5-4468-ad35-45ea04233628 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:07.391858 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d5971-70b5-4468-ad35-45ea04233628 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:07.392124 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d5971-70b5-4468-ad35-45ea04233628 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:07.392788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d5971-70b5-4468-ad35-45ea04233628 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:07.393085 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d5971-70b5-4468-ad35-45ea04233628 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:07.394516 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d5971-70b5-4468-ad35-45ea04233628 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:07.406827 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 96/197] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:23:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:07.410662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d5971-70b5-4468-ad35-45ea04233628 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:07.435420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a725c5b-e63c-4734-8700-9bacec1d8e95 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:07.480833 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bab96a-cf2f-49f4-8340-3a63f010054e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:07.481799 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bab96a-cf2f-49f4-8340-3a63f010054e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:07.482276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bab96a-cf2f-49f4-8340-3a63f010054e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:07.482722 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bab96a-cf2f-49f4-8340-3a63f010054e None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:07.483713 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bab96a-cf2f-49f4-8340-3a63f010054e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:07.484301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bab96a-cf2f-49f4-8340-3a63f010054e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:07.485901 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bab96a-cf2f-49f4-8340-3a63f010054e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:07.495206 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 102/198] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 08:23:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:07.496378 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bab96a-cf2f-49f4-8340-3a63f010054e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:07.536721 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d5971-70b5-4468-ad35-45ea04233628 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:07.583328 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37607346-4dbb-4d7c-92db-c39469717237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:07.583942 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37607346-4dbb-4d7c-92db-c39469717237 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:07.584170 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37607346-4dbb-4d7c-92db-c39469717237 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:07.584438 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37607346-4dbb-4d7c-92db-c39469717237 None admin] PATH_INFO: `/v3/projects/3ab93f396edd405c96c6effd6d5edb41` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:07.585922 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37607346-4dbb-4d7c-92db-c39469717237 None admin] RBAC: Authorizing `identity:get_project(project_id=3ab93f396edd405c96c6effd6d5edb41)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:07.587622 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37607346-4dbb-4d7c-92db-c39469717237 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:07.593228 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 97/199] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:07 2022] GET /identity/v3/projects/3ab93f396edd405c96c6effd6d5edb41 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:07.593228 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37607346-4dbb-4d7c-92db-c39469717237 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:07.614490 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bab96a-cf2f-49f4-8340-3a63f010054e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:07.645294 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:07.650751 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d61294-37a1-4e82-9933-f7c73c63a944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:07.651631 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d61294-37a1-4e82-9933-f7c73c63a944 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:07.652111 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d61294-37a1-4e82-9933-f7c73c63a944 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:07.652577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d61294-37a1-4e82-9933-f7c73c63a944 None admin] PATH_INFO: `/v3/users/b369df7c929c476ebfc35f907c63d466` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:07.654054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d61294-37a1-4e82-9933-f7c73c63a944 None admin] RBAC: Authorizing `identity:get_user(user_id=b369df7c929c476ebfc35f907c63d466)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:07.655870 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d61294-37a1-4e82-9933-f7c73c63a944 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:07.657851 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 103/200] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:23:07 2022] GET /identity/v3/users/b369df7c929c476ebfc35f907c63d466 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:07.659076 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d61294-37a1-4e82-9933-f7c73c63a944 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:07.712357 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37607346-4dbb-4d7c-92db-c39469717237 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:07.738594 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:07.758489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30335a90-46f7-44b3-9204-330769b09406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:07.759123 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30335a90-46f7-44b3-9204-330769b09406 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:07.759397 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30335a90-46f7-44b3-9204-330769b09406 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:07.759800 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30335a90-46f7-44b3-9204-330769b09406 None admin] PATH_INFO: `/v3/users/becd3f0974834999b872e1cbbe9a4d6b` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:07.761315 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30335a90-46f7-44b3-9204-330769b09406 None admin] RBAC: Authorizing `identity:get_user(user_id=becd3f0974834999b872e1cbbe9a4d6b)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:07.763351 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30335a90-46f7-44b3-9204-330769b09406 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:07.766821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 98/201] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:23:07 2022] GET /identity/v3/users/becd3f0974834999b872e1cbbe9a4d6b => generated 289 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:07.767709 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30335a90-46f7-44b3-9204-330769b09406 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:07.768664 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d61294-37a1-4e82-9933-f7c73c63a944 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:07.814490 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8eca5-249f-4f79-9972-6fe1d0540246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:07.815109 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8eca5-249f-4f79-9972-6fe1d0540246 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:07.815390 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8eca5-249f-4f79-9972-6fe1d0540246 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:07.815662 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8eca5-249f-4f79-9972-6fe1d0540246 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:07.816948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8eca5-249f-4f79-9972-6fe1d0540246 None admin] RBAC: Adding query filter params (role.id=c1e514a4b5424f959b069f36cbfd8147, scope.project.id=3ab93f396edd405c96c6effd6d5edb41, user.id=b369df7c929c476ebfc35f907c63d466) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:07.817287 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8eca5-249f-4f79-9972-6fe1d0540246 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:07.819195 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8eca5-249f-4f79-9972-6fe1d0540246 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:07.833205 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 104/202] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:23:07 2022] GET /identity/v3/role_assignments?user.id=b369df7c929c476ebfc35f907c63d466&scope.project.id=3ab93f396edd405c96c6effd6d5edb41&role.id=c1e514a4b5424f959b069f36cbfd8147 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:07.833205 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8eca5-249f-4f79-9972-6fe1d0540246 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:07.889968 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:07.946075 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30335a90-46f7-44b3-9204-330769b09406 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:07.950734 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8eca5-249f-4f79-9972-6fe1d0540246 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:07.969026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:07.991903 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96729dd5-8d82-4110-867b-10eaef716d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:07.992799 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96729dd5-8d82-4110-867b-10eaef716d5a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:07.993310 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96729dd5-8d82-4110-867b-10eaef716d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:07.993747 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96729dd5-8d82-4110-867b-10eaef716d5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:08.051322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108d373-7ae1-4ff6-862f-66cab2ba47a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:08.052251 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108d373-7ae1-4ff6-862f-66cab2ba47a8 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:08.052759 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108d373-7ae1-4ff6-862f-66cab2ba47a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:08.053126 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108d373-7ae1-4ff6-862f-66cab2ba47a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:08.096195 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96729dd5-8d82-4110-867b-10eaef716d5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:08.097722 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96729dd5-8d82-4110-867b-10eaef716d5a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:08.182639 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:08.205975 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108d373-7ae1-4ff6-862f-66cab2ba47a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:08.207489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108d373-7ae1-4ff6-862f-66cab2ba47a8 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:08.285676 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 105/203] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:23:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:23:08.286639 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96729dd5-8d82-4110-867b-10eaef716d5a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:08.373246 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 99/204] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:23:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:23:08.395443 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108d373-7ae1-4ff6-862f-66cab2ba47a8 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:08.409127 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:08.496481 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96729dd5-8d82-4110-867b-10eaef716d5a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:08.558703 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3fa04-8611-41fc-9fa8-07fa60816a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:08.559390 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3fa04-8611-41fc-9fa8-07fa60816a0e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:08.559671 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3fa04-8611-41fc-9fa8-07fa60816a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:08.559940 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3fa04-8611-41fc-9fa8-07fa60816a0e None admin] PATH_INFO: `/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:08.562979 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b3fa04-8611-41fc-9fa8-07fa60816a0e None admin] RBAC: Authorizing `identity:get_project(project_id=ce0954e79d28403bb7e0c4f0b78e4394)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:08.564444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b3fa04-8611-41fc-9fa8-07fa60816a0e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:08.566092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 106/205] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:08 2022] GET /identity/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394 => generated 298 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:08.572823 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108d373-7ae1-4ff6-862f-66cab2ba47a8 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:08.578285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3fa04-8611-41fc-9fa8-07fa60816a0e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:08.615715 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b03abd-db9b-471a-b754-93d276ad1434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:08.616411 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b03abd-db9b-471a-b754-93d276ad1434 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:08.616628 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b03abd-db9b-471a-b754-93d276ad1434 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:08.616898 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b03abd-db9b-471a-b754-93d276ad1434 None admin] PATH_INFO: `/v3/projects/3ab93f396edd405c96c6effd6d5edb41` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:08.617763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:08.622221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b03abd-db9b-471a-b754-93d276ad1434 None admin] RBAC: Authorizing `identity:get_project(project_id=3ab93f396edd405c96c6effd6d5edb41)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:08.623683 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b03abd-db9b-471a-b754-93d276ad1434 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:08.625634 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 100/206] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:08 2022] GET /identity/v3/projects/3ab93f396edd405c96c6effd6d5edb41 => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:08.632250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b03abd-db9b-471a-b754-93d276ad1434 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:08.766049 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3fa04-8611-41fc-9fa8-07fa60816a0e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:08.766570 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:08.787831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b03abd-db9b-471a-b754-93d276ad1434 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:08.822300 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4a17d-08bd-421a-a609-183ee5e9885b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:08.823169 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4a17d-08bd-421a-a609-183ee5e9885b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:08.823655 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4a17d-08bd-421a-a609-183ee5e9885b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:08.824114 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4a17d-08bd-421a-a609-183ee5e9885b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:08.825047 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c4a17d-08bd-421a-a609-183ee5e9885b None admin] RBAC: Adding query filter params (role.id=152f4082bb67479ebaeb08c2fa683399, scope.project.id=3ab93f396edd405c96c6effd6d5edb41, user.id=b369df7c929c476ebfc35f907c63d466) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:08.825692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c4a17d-08bd-421a-a609-183ee5e9885b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:08.827320 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c4a17d-08bd-421a-a609-183ee5e9885b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:08.834141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 101/207] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:23:08 2022] GET /identity/v3/role_assignments?user.id=b369df7c929c476ebfc35f907c63d466&scope.project.id=3ab93f396edd405c96c6effd6d5edb41&role.id=152f4082bb67479ebaeb08c2fa683399 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:08.854325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0d0bd-ad0c-42ae-9856-21796a5735bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:08.854961 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0d0bd-ad0c-42ae-9856-21796a5735bc None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:08.855240 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0d0bd-ad0c-42ae-9856-21796a5735bc None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:08.855509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0d0bd-ad0c-42ae-9856-21796a5735bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:08.856257 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd0d0bd-ad0c-42ae-9856-21796a5735bc None admin] RBAC: Adding query filter params (role.id=ab682c3121c04a5fb600dc47750d013c, scope.project.id=ce0954e79d28403bb7e0c4f0b78e4394, user.id=becd3f0974834999b872e1cbbe9a4d6b) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:08.856560 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd0d0bd-ad0c-42ae-9856-21796a5735bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:08.858064 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd0d0bd-ad0c-42ae-9856-21796a5735bc None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:08.870791 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 107/208] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:23:08 2022] GET /identity/v3/role_assignments?user.id=becd3f0974834999b872e1cbbe9a4d6b&scope.project.id=ce0954e79d28403bb7e0c4f0b78e4394&role.id=ab682c3121c04a5fb600dc47750d013c => generated 608 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:09.176062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8cc97-b14b-4cd2-bf18-10c7ad1dd9dc None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:09.176447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8cc97-b14b-4cd2-bf18-10c7ad1dd9dc None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:09.176645 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8cc97-b14b-4cd2-bf18-10c7ad1dd9dc None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:09.178381 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 102/209] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:09.184095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03e035-e5a2-4c38-b5b9-58c0fb66f901 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:09.184395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03e035-e5a2-4c38-b5b9-58c0fb66f901 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:09.184651 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03e035-e5a2-4c38-b5b9-58c0fb66f901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:09.206254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-2b03e035-e5a2-4c38-b5b9-58c0fb66f901 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:09.207524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2b03e035-e5a2-4c38-b5b9-58c0fb66f901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:09.282270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 108/210] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:09.305584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8cc97-b14b-4cd2-bf18-10c7ad1dd9dc None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:09.380181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c728aad-be1c-4e4a-ac4a-3273eec4173c None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:09.380462 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c728aad-be1c-4e4a-ac4a-3273eec4173c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:09.380736 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c728aad-be1c-4e4a-ac4a-3273eec4173c None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:09.382369 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 109/211] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 08:23:09.393467 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022742c2-2e7b-42d9-bb7d-e785febdb423 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:09.393743 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022742c2-2e7b-42d9-bb7d-e785febdb423 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:09.394019 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022742c2-2e7b-42d9-bb7d-e785febdb423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:09.413926 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-022742c2-2e7b-42d9-bb7d-e785febdb423 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:09.415890 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-022742c2-2e7b-42d9-bb7d-e785febdb423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:09.426764 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:09.428043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8cc97-b14b-4cd2-bf18-10c7ad1dd9dc None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:09.476435 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf33b7f-baac-4e51-9c5b-ce9658bab3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:09.477259 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf33b7f-baac-4e51-9c5b-ce9658bab3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:09.477545 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf33b7f-baac-4e51-9c5b-ce9658bab3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:09.477820 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf33b7f-baac-4e51-9c5b-ce9658bab3f5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:09.491215 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf33b7f-baac-4e51-9c5b-ce9658bab3f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:09.496069 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-3cf33b7f-baac-4e51-9c5b-ce9658bab3f5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:23:09.496069 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:09.496069 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:09.496069 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:09.496069 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:09.496069 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:09.496069 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:09.496069 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:09.496069 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:09.496069 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:09.496069 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:09.496069 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:09.496069 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:09.496069 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:09.496069 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:09.496069 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:09.496959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:09.496959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:09.496959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:09.496959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:09.496959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:09.496959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:09.496959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:09.496959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:09.496959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:09.496959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:09.496959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:09.496959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:09.496959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:09.496959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:09.496959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:09.496959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:09.497795 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:09.497795 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:09.497795 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:09.497795 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:09.497795 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:09.497795 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:09.497795 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:09.497795 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:09.497795 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:09.497795 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:09.497795 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:23:09.497795 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:09.497795 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 103/212] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:09.504569 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf33b7f-baac-4e51-9c5b-ce9658bab3f5 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:09.508543 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:09.512727 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 110/213] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:23:09.537224 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022742c2-2e7b-42d9-bb7d-e785febdb423 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:09.617955 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf33b7f-baac-4e51-9c5b-ce9658bab3f5 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:09.647943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022742c2-2e7b-42d9-bb7d-e785febdb423 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:09.654987 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa33e9d-11b5-4172-8242-fab1b63089b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:09.655600 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa33e9d-11b5-4172-8242-fab1b63089b2 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:09.656450 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa33e9d-11b5-4172-8242-fab1b63089b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:09.656944 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa33e9d-11b5-4172-8242-fab1b63089b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:09.657739 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa33e9d-11b5-4172-8242-fab1b63089b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:09.658040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa33e9d-11b5-4172-8242-fab1b63089b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:09.659481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa33e9d-11b5-4172-8242-fab1b63089b2 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:09.668311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 104/214] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 08:23:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:09.674474 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa33e9d-11b5-4172-8242-fab1b63089b2 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:09.685273 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7af5d1-6e5e-40d1-ba70-1c38f5a577ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:09.685881 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7af5d1-6e5e-40d1-ba70-1c38f5a577ce None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:09.686231 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7af5d1-6e5e-40d1-ba70-1c38f5a577ce None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:09.686508 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7af5d1-6e5e-40d1-ba70-1c38f5a577ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:09.687882 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7af5d1-6e5e-40d1-ba70-1c38f5a577ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:09.689352 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7af5d1-6e5e-40d1-ba70-1c38f5a577ce None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:09.691318 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 111/215] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:09.696608 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7af5d1-6e5e-40d1-ba70-1c38f5a577ce None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:09.709776 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:09.746404 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:09.786079 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa33e9d-11b5-4172-8242-fab1b63089b2 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:09.800469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7af5d1-6e5e-40d1-ba70-1c38f5a577ce None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:09.823381 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b15250-cad9-463d-b694-9384ba77b6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:09.824018 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b15250-cad9-463d-b694-9384ba77b6f9 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:09.824299 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b15250-cad9-463d-b694-9384ba77b6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:09.824705 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b15250-cad9-463d-b694-9384ba77b6f9 None admin] PATH_INFO: `/v3/users/a585538b793e490d998d7cf6ffb2d8f8` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:09.837978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc23a7f9-43d1-4064-bac0-8775de2ead95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:09.838868 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23a7f9-43d1-4064-bac0-8775de2ead95 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:09.839445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23a7f9-43d1-4064-bac0-8775de2ead95 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:09.839617 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b15250-cad9-463d-b694-9384ba77b6f9 None admin] RBAC: Authorizing `identity:get_user(user_id=a585538b793e490d998d7cf6ffb2d8f8)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:09.840075 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23a7f9-43d1-4064-bac0-8775de2ead95 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:09.840919 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc23a7f9-43d1-4064-bac0-8775de2ead95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:09.841788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b15250-cad9-463d-b694-9384ba77b6f9 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:09.842776 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc23a7f9-43d1-4064-bac0-8775de2ead95 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:09.844005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 105/216] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:23:09 2022] GET /identity/v3/users/a585538b793e490d998d7cf6ffb2d8f8 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:09.849886 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b15250-cad9-463d-b694-9384ba77b6f9 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:09.949668 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 112/217] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 08:23:09 2022] POST /identity/v3/users => generated 297 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:23:09.967691 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b15250-cad9-463d-b694-9384ba77b6f9 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:09.997031 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:10.010614 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab26d102-158a-4a1f-b598-1c7a55436d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:10.013157 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26d102-158a-4a1f-b598-1c7a55436d02 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:10.013157 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26d102-158a-4a1f-b598-1c7a55436d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:10.013157 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26d102-158a-4a1f-b598-1c7a55436d02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:10.092300 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab26d102-158a-4a1f-b598-1c7a55436d02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:10.094500 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab26d102-158a-4a1f-b598-1c7a55436d02 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:10.121865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c53f72-a3ea-4c59-8c81-1e2033bdd1eb None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:10.122191 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c53f72-a3ea-4c59-8c81-1e2033bdd1eb None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:10.122421 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c53f72-a3ea-4c59-8c81-1e2033bdd1eb None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:10.124065 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 113/218] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 08:23:10.129511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9622f-d78e-44a6-85a3-7d0a423061bb None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:10.129796 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9622f-d78e-44a6-85a3-7d0a423061bb None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:10.130074 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9622f-d78e-44a6-85a3-7d0a423061bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:10.153569 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-17b9622f-d78e-44a6-85a3-7d0a423061bb None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:10.156408 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-17b9622f-d78e-44a6-85a3-7d0a423061bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:10.219736 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 106/219] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:23:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 08:23:10.226039 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab26d102-158a-4a1f-b598-1c7a55436d02 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:10.226758 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:10.237985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 114/220] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:23:10.256380 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:10.256966 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b9622f-d78e-44a6-85a3-7d0a423061bb None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:10.351239 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab26d102-158a-4a1f-b598-1c7a55436d02 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:10.384213 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b9622f-d78e-44a6-85a3-7d0a423061bb None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:10.398601 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6eb531-3c2f-4367-97f7-f1b6e5c6a6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:10.399196 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6eb531-3c2f-4367-97f7-f1b6e5c6a6ce None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:10.399732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6eb531-3c2f-4367-97f7-f1b6e5c6a6ce None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:10.400009 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6eb531-3c2f-4367-97f7-f1b6e5c6a6ce None admin] PATH_INFO: `/v3/projects/3ab93f396edd405c96c6effd6d5edb41` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:10.401932 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6eb531-3c2f-4367-97f7-f1b6e5c6a6ce None admin] RBAC: Authorizing `identity:get_project(project_id=3ab93f396edd405c96c6effd6d5edb41)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:10.403667 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6eb531-3c2f-4367-97f7-f1b6e5c6a6ce None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:10.406063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 107/221] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:10 2022] GET /identity/v3/projects/3ab93f396edd405c96c6effd6d5edb41 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:10.416492 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6eb531-3c2f-4367-97f7-f1b6e5c6a6ce None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:10.432105 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aca0f2-49e3-4c10-aa8f-ecf66e58ff00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:10.432760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aca0f2-49e3-4c10-aa8f-ecf66e58ff00 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:10.433044 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aca0f2-49e3-4c10-aa8f-ecf66e58ff00 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:10.433345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aca0f2-49e3-4c10-aa8f-ecf66e58ff00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:10.448076 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aca0f2-49e3-4c10-aa8f-ecf66e58ff00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:10.455456 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-02aca0f2-49e3-4c10-aa8f-ecf66e58ff00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:23:10.455456 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:10.455456 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:10.455456 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:10.455456 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:10.455456 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:10.455456 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:10.455456 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:10.455456 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:10.455456 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:10.455456 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:10.455456 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:10.455456 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:10.455456 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:10.455456 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:10.455456 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:10.456397 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:10.456397 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:10.456397 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:10.456397 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:10.456397 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:10.456397 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:10.456397 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:10.456397 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:10.456397 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:10.456397 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:10.456397 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:10.456397 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:10.456397 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:10.456397 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:10.456397 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:10.456397 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:10.457542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:10.457542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:10.457542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:10.457542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:10.457542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:10.457542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:10.457542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:10.457542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:10.457542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:10.457542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:10.457542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:23:10.457542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:10.457542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 115/222] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:23:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:23:10.463064 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aca0f2-49e3-4c10-aa8f-ecf66e58ff00 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:10.526084 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:10.537555 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:10.549269 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6eb531-3c2f-4367-97f7-f1b6e5c6a6ce None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:10.576911 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aca0f2-49e3-4c10-aa8f-ecf66e58ff00 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:10.595695 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a2d9a-b35e-48e6-9ffe-15e5aa80a886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:10.596711 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a2d9a-b35e-48e6-9ffe-15e5aa80a886 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:10.597202 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a2d9a-b35e-48e6-9ffe-15e5aa80a886 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:10.598141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a2d9a-b35e-48e6-9ffe-15e5aa80a886 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:10.599035 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a2d9a-b35e-48e6-9ffe-15e5aa80a886 None admin] RBAC: Adding query filter params (role.id=b0ba6e2bf793402abe2e680c2a7dcb53, scope.project.id=3ab93f396edd405c96c6effd6d5edb41, user.id=a585538b793e490d998d7cf6ffb2d8f8) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:10.599495 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a2d9a-b35e-48e6-9ffe-15e5aa80a886 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:10.601606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a2d9a-b35e-48e6-9ffe-15e5aa80a886 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:10.617181 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 108/223] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:23:10 2022] GET /identity/v3/role_assignments?user.id=a585538b793e490d998d7cf6ffb2d8f8&scope.project.id=3ab93f396edd405c96c6effd6d5edb41&role.id=b0ba6e2bf793402abe2e680c2a7dcb53 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:10.632871 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05808050-a007-436e-9b84-fa9ad07f0ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:10.633541 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05808050-a007-436e-9b84-fa9ad07f0ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:10.633822 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05808050-a007-436e-9b84-fa9ad07f0ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:10.634094 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05808050-a007-436e-9b84-fa9ad07f0ba9 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:10.634784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05808050-a007-436e-9b84-fa9ad07f0ba9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:10.635086 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05808050-a007-436e-9b84-fa9ad07f0ba9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:10.636513 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05808050-a007-436e-9b84-fa9ad07f0ba9 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:10.655619 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 116/224] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:10.661890 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a2d9a-b35e-48e6-9ffe-15e5aa80a886 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:10.820789 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a2d9a-b35e-48e6-9ffe-15e5aa80a886 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:10.857309 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614dc5c-79d3-4859-a804-facfddf8521e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:10.857945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614dc5c-79d3-4859-a804-facfddf8521e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:10.858219 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614dc5c-79d3-4859-a804-facfddf8521e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:10.858486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614dc5c-79d3-4859-a804-facfddf8521e None admin] PATH_INFO: `/v3/users/b369df7c929c476ebfc35f907c63d466` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:10.860121 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0614dc5c-79d3-4859-a804-facfddf8521e None admin] RBAC: Authorizing `identity:get_user(user_id=b369df7c929c476ebfc35f907c63d466)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:10.861601 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0614dc5c-79d3-4859-a804-facfddf8521e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:10.863425 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 109/225] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:23:10 2022] GET /identity/v3/users/b369df7c929c476ebfc35f907c63d466 => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:10.868731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05808050-a007-436e-9b84-fa9ad07f0ba9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:11.002992 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a3109-5fb7-45d0-892a-76d8962c184c None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:11.003338 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a3109-5fb7-45d0-892a-76d8962c184c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:11.003522 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a3109-5fb7-45d0-892a-76d8962c184c None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:11.004737 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05808050-a007-436e-9b84-fa9ad07f0ba9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:11.005109 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 110/226] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 08:23:11.010058 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ba5b2-8be2-4219-ad8e-ee55367d1f69 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:11.010338 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ba5b2-8be2-4219-ad8e-ee55367d1f69 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:11.010613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ba5b2-8be2-4219-ad8e-ee55367d1f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:11.040080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-ce8ba5b2-8be2-4219-ad8e-ee55367d1f69 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:11.041282 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ce8ba5b2-8be2-4219-ad8e-ee55367d1f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:11.045567 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:11.065934 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:11.071299 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b659282-f183-46d8-969d-bf20e56a82e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:11.071594 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b659282-f183-46d8-969d-bf20e56a82e9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:11.071822 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b659282-f183-46d8-969d-bf20e56a82e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:11.072106 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b659282-f183-46d8-969d-bf20e56a82e9 None admin] PATH_INFO: `/v3/projects/3ab93f396edd405c96c6effd6d5edb41` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:11.075982 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b659282-f183-46d8-969d-bf20e56a82e9 None admin] RBAC: Authorizing `identity:get_project(project_id=3ab93f396edd405c96c6effd6d5edb41)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:11.078099 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b659282-f183-46d8-969d-bf20e56a82e9 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:11.079985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 117/227] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:10 2022] GET /identity/v3/projects/3ab93f396edd405c96c6effd6d5edb41 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:11.093177 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b659282-f183-46d8-969d-bf20e56a82e9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:11.153708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 111/228] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:23:11.177373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ba5b2-8be2-4219-ad8e-ee55367d1f69 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:11.241846 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b659282-f183-46d8-969d-bf20e56a82e9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:11.293596 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ba5b2-8be2-4219-ad8e-ee55367d1f69 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:11.309214 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5d082-a30f-4bb6-92d6-8c2f4ba8a870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:11.309994 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5d082-a30f-4bb6-92d6-8c2f4ba8a870 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:11.310272 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5d082-a30f-4bb6-92d6-8c2f4ba8a870 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:11.310544 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5d082-a30f-4bb6-92d6-8c2f4ba8a870 None admin] PATH_INFO: `/v3/projects/3ab93f396edd405c96c6effd6d5edb41/users/b369df7c929c476ebfc35f907c63d466/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:11.318657 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5d082-a30f-4bb6-92d6-8c2f4ba8a870 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ab93f396edd405c96c6effd6d5edb41, user_id=b369df7c929c476ebfc35f907c63d466, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:11.320372 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5d082-a30f-4bb6-92d6-8c2f4ba8a870 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:11.332011 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b83c446-33da-43a1-80fa-8be477c55894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:11.332803 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b83c446-33da-43a1-80fa-8be477c55894 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:11.333167 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b83c446-33da-43a1-80fa-8be477c55894 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:11.333542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b83c446-33da-43a1-80fa-8be477c55894 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:11.345237 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b83c446-33da-43a1-80fa-8be477c55894 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:11.350100 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-3b83c446-33da-43a1-80fa-8be477c55894 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 08:23:11.350100 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:11.350100 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:11.350100 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:11.350100 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:11.350100 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:11.350100 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:11.350100 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:11.350100 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:11.350100 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:11.350100 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:11.350100 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:11.350100 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:11.350100 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:11.350100 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:11.350100 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:11.354447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:11.354447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:11.354447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:11.354447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:11.354447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:11.354447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:11.354447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:11.354447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:11.354447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:11.354447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:11.354447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:11.354447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:11.354447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:11.354447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:11.354447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:11.354447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:11.355704 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:11.355704 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:11.355704 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:11.355704 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:11.355704 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:11.355704 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:11.355704 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:11.355704 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:11.355704 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:11.355704 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:11.355704 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 08:23:11.355704 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:11.355704 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 112/229] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 08:23:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:11.357263 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b83c446-33da-43a1-80fa-8be477c55894 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:11.372994 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:11.427304 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 118/230] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 08:23:11 2022] PUT /identity/v3/projects/3ab93f396edd405c96c6effd6d5edb41/users/b369df7c929c476ebfc35f907c63d466/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:23:11.495409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b83c446-33da-43a1-80fa-8be477c55894 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:11.551060 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6899de2-f7e7-4b27-8aff-896f5410bf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:11.551681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6899de2-f7e7-4b27-8aff-896f5410bf7f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:11.551955 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6899de2-f7e7-4b27-8aff-896f5410bf7f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:11.552223 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6899de2-f7e7-4b27-8aff-896f5410bf7f None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:11.552907 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6899de2-f7e7-4b27-8aff-896f5410bf7f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:11.553234 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6899de2-f7e7-4b27-8aff-896f5410bf7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:11.554638 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6899de2-f7e7-4b27-8aff-896f5410bf7f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:11.564776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 113/231] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:23:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:11.570265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5d082-a30f-4bb6-92d6-8c2f4ba8a870 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:11.641446 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:11.701300 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5d082-a30f-4bb6-92d6-8c2f4ba8a870 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:11.737086 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e5a916-4204-4c56-9b03-133fc6e95728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:11.737795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5a916-4204-4c56-9b03-133fc6e95728 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:11.738079 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5a916-4204-4c56-9b03-133fc6e95728 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:11.738352 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5a916-4204-4c56-9b03-133fc6e95728 None admin] PATH_INFO: `/v3/users/b369df7c929c476ebfc35f907c63d466` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:11.739594 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e5a916-4204-4c56-9b03-133fc6e95728 None admin] RBAC: Authorizing `identity:get_user(user_id=b369df7c929c476ebfc35f907c63d466)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:11.741060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e5a916-4204-4c56-9b03-133fc6e95728 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:11.742969 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 119/232] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:23:11 2022] GET /identity/v3/users/b369df7c929c476ebfc35f907c63d466 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:11.748443 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6899de2-f7e7-4b27-8aff-896f5410bf7f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:11.855322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:11.888034 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6899de2-f7e7-4b27-8aff-896f5410bf7f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:11.933811 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f5e8f-be88-4514-883d-b37401a146a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:11.934377 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f5e8f-be88-4514-883d-b37401a146a6 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:11.934660 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f5e8f-be88-4514-883d-b37401a146a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:11.934937 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f5e8f-be88-4514-883d-b37401a146a6 None admin] PATH_INFO: `/v3/projects/3ab93f396edd405c96c6effd6d5edb41` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:11.936160 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f5e8f-be88-4514-883d-b37401a146a6 None admin] RBAC: Authorizing `identity:get_project(project_id=3ab93f396edd405c96c6effd6d5edb41)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:11.938432 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f5e8f-be88-4514-883d-b37401a146a6 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:11.940674 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 114/233] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:11 2022] GET /identity/v3/projects/3ab93f396edd405c96c6effd6d5edb41 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:11.946316 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e5a916-4204-4c56-9b03-133fc6e95728 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:12.083904 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e5a916-4204-4c56-9b03-133fc6e95728 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:12.100292 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:12.126812 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a162d5-8e3a-404b-ae6d-f0bb2e98b24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:12.127476 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a162d5-8e3a-404b-ae6d-f0bb2e98b24b None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:12.127755 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a162d5-8e3a-404b-ae6d-f0bb2e98b24b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:12.128027 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a162d5-8e3a-404b-ae6d-f0bb2e98b24b None admin] PATH_INFO: `/v3/projects/3ab93f396edd405c96c6effd6d5edb41/users/b369df7c929c476ebfc35f907c63d466/roles/152f4082bb67479ebaeb08c2fa683399` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:12.130302 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a162d5-8e3a-404b-ae6d-f0bb2e98b24b None admin] RBAC: Authorizing `identity:create_grant(project_id=3ab93f396edd405c96c6effd6d5edb41, user_id=b369df7c929c476ebfc35f907c63d466, role_id=152f4082bb67479ebaeb08c2fa683399)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:12.131744 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a162d5-8e3a-404b-ae6d-f0bb2e98b24b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:12.240678 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 120/234] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 08:23:11 2022] PUT /identity/v3/projects/3ab93f396edd405c96c6effd6d5edb41/users/b369df7c929c476ebfc35f907c63d466/roles/152f4082bb67479ebaeb08c2fa683399 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:23:12.707968 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce6ac9-7de7-4cee-976e-a45127f4c2e9 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:12.708367 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce6ac9-7de7-4cee-976e-a45127f4c2e9 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:12.708544 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce6ac9-7de7-4cee-976e-a45127f4c2e9 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:12.710178 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 115/235] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:12.715488 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3fd9a-a30b-447a-9a56-f00126802144 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:12.716020 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3fd9a-a30b-447a-9a56-f00126802144 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:12.716508 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3fd9a-a30b-447a-9a56-f00126802144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:12.746125 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-f7a3fd9a-a30b-447a-9a56-f00126802144 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:12.747619 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f7a3fd9a-a30b-447a-9a56-f00126802144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:12.834755 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 121/236] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:12.865274 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c392eb7-8bdf-4190-8ed5-4d27ae33a425 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:12.865805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c392eb7-8bdf-4190-8ed5-4d27ae33a425 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:12.865805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c392eb7-8bdf-4190-8ed5-4d27ae33a425 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:12.867226 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 116/237] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:12.870946 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3fd9a-a30b-447a-9a56-f00126802144 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:12.872385 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a32f77-dd9d-4cbf-91f7-251bd3d247ba None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:12.872875 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a32f77-dd9d-4cbf-91f7-251bd3d247ba None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:12.873430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a32f77-dd9d-4cbf-91f7-251bd3d247ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:12.898193 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-d4a32f77-dd9d-4cbf-91f7-251bd3d247ba None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:12.899867 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d4a32f77-dd9d-4cbf-91f7-251bd3d247ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:13.026596 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 117/238] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:13.034628 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7d92b-50a0-443e-b529-66276ed4fa18 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:13.035231 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7d92b-50a0-443e-b529-66276ed4fa18 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:13.035734 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7d92b-50a0-443e-b529-66276ed4fa18 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:13.037190 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 118/239] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:13.044916 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb283d88-06d5-487b-a568-09364913668e None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:13.045315 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb283d88-06d5-487b-a568-09364913668e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:13.045663 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb283d88-06d5-487b-a568-09364913668e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:13.046970 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3fd9a-a30b-447a-9a56-f00126802144 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:13.050624 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:13.077376 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-eb283d88-06d5-487b-a568-09364913668e None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:13.078835 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-eb283d88-06d5-487b-a568-09364913668e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:13.090646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31bcc8b-562e-4b56-bb82-654c0264ecf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:13.091656 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31bcc8b-562e-4b56-bb82-654c0264ecf0 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:13.094971 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31bcc8b-562e-4b56-bb82-654c0264ecf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:13.094971 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31bcc8b-562e-4b56-bb82-654c0264ecf0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:13.104525 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31bcc8b-562e-4b56-bb82-654c0264ecf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:13.110554 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-e31bcc8b-562e-4b56-bb82-654c0264ecf0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 08:23:13.110554 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:13.110554 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:13.110554 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:13.110554 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:13.110554 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:13.110554 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:13.110554 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:13.110554 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:13.110554 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:13.110554 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:13.110554 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:13.110554 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:13.110554 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:13.110554 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:13.110554 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:13.112030 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:13.112030 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:13.112030 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:13.112030 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:13.112030 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:13.112030 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:13.112030 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:13.112030 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:13.112030 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:13.112030 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:13.112030 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:13.112030 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:13.112030 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:13.112030 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:13.112030 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:13.112030 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:13.113015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:13.113015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:13.113015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:13.113015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:13.113015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:13.113015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:13.113015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:13.113015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:13.113015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:13.113015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:13.113015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 08:23:13.113015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:13.114538 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 122/240] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:13.115281 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31bcc8b-562e-4b56-bb82-654c0264ecf0 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:13.163733 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 119/241] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:13.164700 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb283d88-06d5-487b-a568-09364913668e None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:13.167082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:13.226005 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31bcc8b-562e-4b56-bb82-654c0264ecf0 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:13.262259 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394057bb-9959-4fae-9841-e648a32178db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:13.263223 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394057bb-9959-4fae-9841-e648a32178db None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:13.263702 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394057bb-9959-4fae-9841-e648a32178db None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:13.264293 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394057bb-9959-4fae-9841-e648a32178db None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:13.268445 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb283d88-06d5-487b-a568-09364913668e None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:13.269721 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394057bb-9959-4fae-9841-e648a32178db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:13.272560 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394057bb-9959-4fae-9841-e648a32178db None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:13.274382 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 123/242] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:13.279800 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394057bb-9959-4fae-9841-e648a32178db None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:13.313508 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:13.316433 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01511683-4881-4432-9c3d-de6081362d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:13.317051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01511683-4881-4432-9c3d-de6081362d10 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:13.317359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01511683-4881-4432-9c3d-de6081362d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:13.317631 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01511683-4881-4432-9c3d-de6081362d10 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:13.318303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01511683-4881-4432-9c3d-de6081362d10 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:13.318604 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01511683-4881-4432-9c3d-de6081362d10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:13.319978 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01511683-4881-4432-9c3d-de6081362d10 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:13.329006 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 120/243] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 08:23:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:13.329755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01511683-4881-4432-9c3d-de6081362d10 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:13.393879 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:13.406427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394057bb-9959-4fae-9841-e648a32178db None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:13.433768 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01511683-4881-4432-9c3d-de6081362d10 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:13.447037 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51bed61-9e0d-47d7-b63e-fcecce3c435d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:13.447858 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51bed61-9e0d-47d7-b63e-fcecce3c435d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:13.448361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51bed61-9e0d-47d7-b63e-fcecce3c435d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:13.448757 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51bed61-9e0d-47d7-b63e-fcecce3c435d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:13.460507 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51bed61-9e0d-47d7-b63e-fcecce3c435d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:13.466395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-a51bed61-9e0d-47d7-b63e-fcecce3c435d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:23:13.466395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:13.466395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:13.466395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:13.466395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:13.466395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:13.466395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:13.466395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:13.466395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:13.466395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:13.466395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:13.466395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:13.466395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:13.466395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:13.466395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:13.466395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:13.467487 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:13.467487 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:13.467487 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:13.467487 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:13.467487 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:13.467487 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:13.467487 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:13.467487 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:13.467487 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:13.467487 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:13.467487 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:13.467487 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:13.467487 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:13.467487 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:13.467487 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:13.467487 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:13.468630 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:13.468630 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:13.468630 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:13.468630 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:13.468630 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:13.468630 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:13.468630 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:13.468630 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:13.468630 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:13.468630 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:13.468630 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:23:13.468630 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:13.469632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 124/244] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:13.470509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51bed61-9e0d-47d7-b63e-fcecce3c435d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:13.473487 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8bb031-187e-4907-ab28-ae7deff3b251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:13.474255 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8bb031-187e-4907-ab28-ae7deff3b251 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:13.474598 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8bb031-187e-4907-ab28-ae7deff3b251 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:13.474992 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8bb031-187e-4907-ab28-ae7deff3b251 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:13.475709 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8bb031-187e-4907-ab28-ae7deff3b251 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:13.477366 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8bb031-187e-4907-ab28-ae7deff3b251 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:13.567764 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:13.576157 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51bed61-9e0d-47d7-b63e-fcecce3c435d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:13.582912 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 121/245] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 08:23:13 2022] POST /identity/v3/users => generated 293 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:23:13.592999 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8bb031-187e-4907-ab28-ae7deff3b251 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:13.614952 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeda598-5b92-40ce-aa71-e92fdad6b534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:13.615723 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeda598-5b92-40ce-aa71-e92fdad6b534 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:13.616009 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeda598-5b92-40ce-aa71-e92fdad6b534 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:13.616282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeda598-5b92-40ce-aa71-e92fdad6b534 None admin] PATH_INFO: `/v3/users/0578f9ae72c54a05adcf7de0ae06ca36` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:13.631960 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeda598-5b92-40ce-aa71-e92fdad6b534 None admin] RBAC: Authorizing `identity:get_user(user_id=0578f9ae72c54a05adcf7de0ae06ca36)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:13.633482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeda598-5b92-40ce-aa71-e92fdad6b534 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:13.639906 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 125/246] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:23:13 2022] GET /identity/v3/users/0578f9ae72c54a05adcf7de0ae06ca36 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:13.642008 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeda598-5b92-40ce-aa71-e92fdad6b534 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:13.778810 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8bb031-187e-4907-ab28-ae7deff3b251 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:13.806234 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:13.822055 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeda598-5b92-40ce-aa71-e92fdad6b534 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:13.825198 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a7d88-b9f1-469a-80a2-07536823ea59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:13.825913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a7d88-b9f1-469a-80a2-07536823ea59 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:13.826267 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a7d88-b9f1-469a-80a2-07536823ea59 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:13.826615 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a7d88-b9f1-469a-80a2-07536823ea59 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:13.827369 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a7d88-b9f1-469a-80a2-07536823ea59 None admin] RBAC: Adding query filter params (name=member) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:13.827803 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a7d88-b9f1-469a-80a2-07536823ea59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:13.830560 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a7d88-b9f1-469a-80a2-07536823ea59 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:13.843752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 122/247] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 08:23:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:13.851442 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a7d88-b9f1-469a-80a2-07536823ea59 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:13.865562 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed7028c-050a-4469-8e11-175c7df450cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:13.866309 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7028c-050a-4469-8e11-175c7df450cc None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:13.866585 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7028c-050a-4469-8e11-175c7df450cc None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:13.866893 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7028c-050a-4469-8e11-175c7df450cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:13.895288 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:13.972668 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed7028c-050a-4469-8e11-175c7df450cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:13.974352 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed7028c-050a-4469-8e11-175c7df450cc None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:13.994129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a7d88-b9f1-469a-80a2-07536823ea59 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:14.044868 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b412d-490f-4216-8110-60e62a85753a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:14.045932 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b412d-490f-4216-8110-60e62a85753a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:14.046721 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b412d-490f-4216-8110-60e62a85753a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:14.047268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b412d-490f-4216-8110-60e62a85753a None admin] PATH_INFO: `/v3/users/a585538b793e490d998d7cf6ffb2d8f8` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:14.048676 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85b412d-490f-4216-8110-60e62a85753a None admin] RBAC: Authorizing `identity:get_user(user_id=a585538b793e490d998d7cf6ffb2d8f8)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:14.050628 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85b412d-490f-4216-8110-60e62a85753a None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:14.052839 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 123/248] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:23:13 2022] GET /identity/v3/users/a585538b793e490d998d7cf6ffb2d8f8 => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:14.055297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cff6b-6e4b-474f-835c-c4a87d056437 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:14.055585 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cff6b-6e4b-474f-835c-c4a87d056437 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:14.055995 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cff6b-6e4b-474f-835c-c4a87d056437 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:14.057524 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 124/249] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:14.059163 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51cff6b-6e4b-474f-835c-c4a87d056437 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:14.106781 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 126/250] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:23:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:23:14.108040 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78062d-f9a6-4ac1-aed9-da645f544582 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:14.108387 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78062d-f9a6-4ac1-aed9-da645f544582 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:14.109346 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78062d-f9a6-4ac1-aed9-da645f544582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:14.127306 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:14.129429 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-5c78062d-f9a6-4ac1-aed9-da645f544582 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:14.130608 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5c78062d-f9a6-4ac1-aed9-da645f544582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:14.184710 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51cff6b-6e4b-474f-835c-c4a87d056437 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:14.186608 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:14.212544 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 127/251] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:14.213439 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c78062d-f9a6-4ac1-aed9-da645f544582 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:14.234404 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165face8-4cd5-4da4-af00-d79eefba034a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:14.235040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165face8-4cd5-4da4-af00-d79eefba034a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:14.235308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165face8-4cd5-4da4-af00-d79eefba034a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:14.235575 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165face8-4cd5-4da4-af00-d79eefba034a None admin] PATH_INFO: `/v3/projects/3ab93f396edd405c96c6effd6d5edb41` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:14.237480 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165face8-4cd5-4da4-af00-d79eefba034a None admin] RBAC: Authorizing `identity:get_project(project_id=3ab93f396edd405c96c6effd6d5edb41)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:14.238904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165face8-4cd5-4da4-af00-d79eefba034a None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:14.243571 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 125/252] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:14 2022] GET /identity/v3/projects/3ab93f396edd405c96c6effd6d5edb41 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:14.247292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165face8-4cd5-4da4-af00-d79eefba034a None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:14.352167 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165face8-4cd5-4da4-af00-d79eefba034a None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:14.370870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:14.388838 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e25a8-437e-4471-9c71-245ec5911626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:14.389533 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e25a8-437e-4471-9c71-245ec5911626 None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:14.389860 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e25a8-437e-4471-9c71-245ec5911626 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:14.390282 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e25a8-437e-4471-9c71-245ec5911626 None admin] PATH_INFO: `/v3/projects/3ab93f396edd405c96c6effd6d5edb41/users/a585538b793e490d998d7cf6ffb2d8f8/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:14.392333 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e25a8-437e-4471-9c71-245ec5911626 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ab93f396edd405c96c6effd6d5edb41, user_id=a585538b793e490d998d7cf6ffb2d8f8, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:14.393811 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e25a8-437e-4471-9c71-245ec5911626 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:14.429894 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c78062d-f9a6-4ac1-aed9-da645f544582 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:14.467176 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 126/253] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 08:23:14 2022] PUT /identity/v3/projects/3ab93f396edd405c96c6effd6d5edb41/users/a585538b793e490d998d7cf6ffb2d8f8/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:23:14.468112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e25a8-437e-4471-9c71-245ec5911626 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:14.497041 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a62fe-6b64-4d24-8a81-4f65c361d397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:14.499133 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a62fe-6b64-4d24-8a81-4f65c361d397 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:14.500234 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a62fe-6b64-4d24-8a81-4f65c361d397 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:14.500503 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a62fe-6b64-4d24-8a81-4f65c361d397 None admin] PATH_INFO: `/v3/projects/3ab93f396edd405c96c6effd6d5edb41` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:14.502540 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a62fe-6b64-4d24-8a81-4f65c361d397 None admin] RBAC: Authorizing `identity:get_project(project_id=3ab93f396edd405c96c6effd6d5edb41)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:14.504019 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a62fe-6b64-4d24-8a81-4f65c361d397 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:14.516726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 128/254] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:14 2022] GET /identity/v3/projects/3ab93f396edd405c96c6effd6d5edb41 => generated 302 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:14.524366 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a62fe-6b64-4d24-8a81-4f65c361d397 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:14.630687 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:14.691196 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a62fe-6b64-4d24-8a81-4f65c361d397 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:14.701922 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e25a8-437e-4471-9c71-245ec5911626 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:14.735275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e66169-0f91-4f12-b0c7-d7ae32da458d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:14.735885 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e66169-0f91-4f12-b0c7-d7ae32da458d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:14.736190 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e66169-0f91-4f12-b0c7-d7ae32da458d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:14.737185 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e66169-0f91-4f12-b0c7-d7ae32da458d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:14.737518 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e66169-0f91-4f12-b0c7-d7ae32da458d None admin] RBAC: Adding query filter params (role.id=ab682c3121c04a5fb600dc47750d013c, scope.project.id=3ab93f396edd405c96c6effd6d5edb41, user.id=0578f9ae72c54a05adcf7de0ae06ca36) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:14.737820 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e66169-0f91-4f12-b0c7-d7ae32da458d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:14.739726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e66169-0f91-4f12-b0c7-d7ae32da458d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:14.746376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 129/255] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:23:14 2022] GET /identity/v3/role_assignments?user.id=0578f9ae72c54a05adcf7de0ae06ca36&scope.project.id=3ab93f396edd405c96c6effd6d5edb41&role.id=ab682c3121c04a5fb600dc47750d013c => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:14.767223 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc782132-1b9b-4268-9b17-534fb8f91912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:14.768207 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc782132-1b9b-4268-9b17-534fb8f91912 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:14.768700 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc782132-1b9b-4268-9b17-534fb8f91912 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:14.769191 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc782132-1b9b-4268-9b17-534fb8f91912 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:14.780414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc782132-1b9b-4268-9b17-534fb8f91912 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:14.786026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-cc782132-1b9b-4268-9b17-534fb8f91912 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:23:14.786026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:14.786026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:14.786026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:14.786026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:14.786026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:14.786026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:14.786026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:14.786026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:14.786026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:14.786026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:14.786026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:14.786026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:14.786026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:14.786026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:14.786026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:14.786899 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:14.786899 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:14.786899 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:14.786899 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:14.786899 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:14.786899 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:14.786899 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:14.786899 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:14.786899 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:14.786899 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:14.786899 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:14.786899 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:14.786899 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:14.786899 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:14.786899 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:14.786899 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:14.787706 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:14.787706 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:14.787706 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:14.787706 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:14.787706 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:14.787706 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:14.787706 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:14.787706 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:14.787706 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:14.787706 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:14.787706 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:23:14.787706 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:14.791949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 127/256] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:23:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:14.797049 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e66169-0f91-4f12-b0c7-d7ae32da458d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:14.915044 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e66169-0f91-4f12-b0c7-d7ae32da458d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:14.929937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:14.951843 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb64d1e-46bb-4b4d-b057-331e680e33c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:14.953064 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb64d1e-46bb-4b4d-b057-331e680e33c2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:14.953605 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb64d1e-46bb-4b4d-b057-331e680e33c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:14.953914 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb64d1e-46bb-4b4d-b057-331e680e33c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:14.954604 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb64d1e-46bb-4b4d-b057-331e680e33c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:14.954904 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb64d1e-46bb-4b4d-b057-331e680e33c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:14.956577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb64d1e-46bb-4b4d-b057-331e680e33c2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:14.966301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 130/257] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:14.972118 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc782132-1b9b-4268-9b17-534fb8f91912 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:15.042371 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:15.110851 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc782132-1b9b-4268-9b17-534fb8f91912 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:15.155053 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cabb5d-3c9c-44c8-8000-b1c56036ee5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:15.156220 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cabb5d-3c9c-44c8-8000-b1c56036ee5c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:15.156507 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cabb5d-3c9c-44c8-8000-b1c56036ee5c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:15.156856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cabb5d-3c9c-44c8-8000-b1c56036ee5c None admin] PATH_INFO: `/v3/users/b369df7c929c476ebfc35f907c63d466` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:15.160948 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cabb5d-3c9c-44c8-8000-b1c56036ee5c None admin] RBAC: Authorizing `identity:get_user(user_id=b369df7c929c476ebfc35f907c63d466)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:15.162461 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cabb5d-3c9c-44c8-8000-b1c56036ee5c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:15.164399 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 128/258] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:23:14 2022] GET /identity/v3/users/b369df7c929c476ebfc35f907c63d466 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:15.170079 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb64d1e-46bb-4b4d-b057-331e680e33c2 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:15.308184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb64d1e-46bb-4b4d-b057-331e680e33c2 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:15.363344 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902a1e5-8b48-41a1-9224-2c80b4d386bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:15.363981 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902a1e5-8b48-41a1-9224-2c80b4d386bc None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:15.364264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902a1e5-8b48-41a1-9224-2c80b4d386bc None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:15.364537 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902a1e5-8b48-41a1-9224-2c80b4d386bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:15.389017 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd699854-a801-4864-9884-7d972b3e0d46 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:15.389333 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd699854-a801-4864-9884-7d972b3e0d46 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:15.389607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd699854-a801-4864-9884-7d972b3e0d46 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:15.393038 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 129/259] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:15.400626 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b2369-dda8-4cdb-b470-1852ef4e9083 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:15.400908 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b2369-dda8-4cdb-b470-1852ef4e9083 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:15.401219 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b2369-dda8-4cdb-b470-1852ef4e9083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:15.441973 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-961b2369-dda8-4cdb-b470-1852ef4e9083 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:15.443833 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:15.444535 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-961b2369-dda8-4cdb-b470-1852ef4e9083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:15.496355 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902a1e5-8b48-41a1-9224-2c80b4d386bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:15.497890 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902a1e5-8b48-41a1-9224-2c80b4d386bc None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:15.602192 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 130/260] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:23:15.653452 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b2369-dda8-4cdb-b470-1852ef4e9083 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:15.679729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 131/261] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:23:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:23:15.685567 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902a1e5-8b48-41a1-9224-2c80b4d386bc None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:15.745745 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:15.840884 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:15.867938 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b2369-dda8-4cdb-b470-1852ef4e9083 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:15.884974 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902a1e5-8b48-41a1-9224-2c80b4d386bc None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:15.929221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d3a0a-2fdd-4bbf-886c-8c3bc7384ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:15.929931 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d3a0a-2fdd-4bbf-886c-8c3bc7384ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:15.930146 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d3a0a-2fdd-4bbf-886c-8c3bc7384ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:15.930420 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d3a0a-2fdd-4bbf-886c-8c3bc7384ea8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:15.941818 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7d3a0a-2fdd-4bbf-886c-8c3bc7384ea8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:15.946892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-0f7d3a0a-2fdd-4bbf-886c-8c3bc7384ea8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 08:23:15.946892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:15.946892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:15.946892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:15.946892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:15.946892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:15.946892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:15.946892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:15.946892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:15.946892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:15.946892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:15.946892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:15.946892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:15.946892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:15.946892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:15.946892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:15.947755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:15.947755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:15.947755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:15.947755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:15.947755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:15.947755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:15.947755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:15.947755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:15.947755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:15.947755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:15.947755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:15.947755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:15.947755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:15.947755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:15.947755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:15.947755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:15.948531 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:15.948531 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:15.948531 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:15.948531 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:15.948531 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:15.948531 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:15.948531 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:15.948531 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:15.948531 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:15.948531 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:15.948531 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 08:23:15.948531 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:15.949115 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b41b1d-577d-4139-8aa3-6a0a272896dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:15.949115 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b41b1d-577d-4139-8aa3-6a0a272896dd None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:15.949278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 131/262] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 08:23:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:23:15.949343 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b41b1d-577d-4139-8aa3-6a0a272896dd None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:15.949343 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b41b1d-577d-4139-8aa3-6a0a272896dd None admin] PATH_INFO: `/v3/projects/3ab93f396edd405c96c6effd6d5edb41` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:15.955056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b41b1d-577d-4139-8aa3-6a0a272896dd None admin] RBAC: Authorizing `identity:get_project(project_id=3ab93f396edd405c96c6effd6d5edb41)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:15.956859 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b41b1d-577d-4139-8aa3-6a0a272896dd None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:15.963550 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 132/263] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:15 2022] GET /identity/v3/projects/3ab93f396edd405c96c6effd6d5edb41 => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:15.965033 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d3a0a-2fdd-4bbf-886c-8c3bc7384ea8 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:15.965525 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b41b1d-577d-4139-8aa3-6a0a272896dd None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:16.079700 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b41b1d-577d-4139-8aa3-6a0a272896dd None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:16.122774 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a3dd2-b71f-44ac-b3cb-b75baa066d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:16.123402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a3dd2-b71f-44ac-b3cb-b75baa066d93 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:16.123680 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a3dd2-b71f-44ac-b3cb-b75baa066d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:16.123953 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a3dd2-b71f-44ac-b3cb-b75baa066d93 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:16.124895 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a3dd2-b71f-44ac-b3cb-b75baa066d93 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:16.125329 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a3dd2-b71f-44ac-b3cb-b75baa066d93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:16.126822 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a3dd2-b71f-44ac-b3cb-b75baa066d93 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:16.135754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 133/264] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:23:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:16.142424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a3dd2-b71f-44ac-b3cb-b75baa066d93 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:16.153983 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:16.160957 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d3a0a-2fdd-4bbf-886c-8c3bc7384ea8 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:16.198201 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52c28c-de6a-40f9-a241-34320a59ced2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:16.198536 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52c28c-de6a-40f9-a241-34320a59ced2 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:16.198750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52c28c-de6a-40f9-a241-34320a59ced2 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:16.199025 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52c28c-de6a-40f9-a241-34320a59ced2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:16.199760 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52c28c-de6a-40f9-a241-34320a59ced2 None admin] RBAC: Adding query filter params (role.id=c1e514a4b5424f959b069f36cbfd8147, scope.project.id=3ab93f396edd405c96c6effd6d5edb41, user.id=b369df7c929c476ebfc35f907c63d466) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:16.200063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52c28c-de6a-40f9-a241-34320a59ced2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:16.201634 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52c28c-de6a-40f9-a241-34320a59ced2 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:16.208821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 132/265] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:23:15 2022] GET /identity/v3/role_assignments?user.id=b369df7c929c476ebfc35f907c63d466&scope.project.id=3ab93f396edd405c96c6effd6d5edb41&role.id=c1e514a4b5424f959b069f36cbfd8147 => generated 608 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:16.225553 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:16.271338 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a3dd2-b71f-44ac-b3cb-b75baa066d93 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:16.307148 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a28dc-abbb-487e-8d92-4b23275a3e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:16.308028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a28dc-abbb-487e-8d92-4b23275a3e0c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:16.308467 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a28dc-abbb-487e-8d92-4b23275a3e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:16.308881 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a28dc-abbb-487e-8d92-4b23275a3e0c None admin] PATH_INFO: `/v3/users/b369df7c929c476ebfc35f907c63d466` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:16.310287 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a28dc-abbb-487e-8d92-4b23275a3e0c None admin] RBAC: Authorizing `identity:get_user(user_id=b369df7c929c476ebfc35f907c63d466)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:16.312514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a28dc-abbb-487e-8d92-4b23275a3e0c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:16.314738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 134/266] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:23:16 2022] GET /identity/v3/users/b369df7c929c476ebfc35f907c63d466 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:16.322636 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52c28c-de6a-40f9-a241-34320a59ced2 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:16.479034 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52c28c-de6a-40f9-a241-34320a59ced2 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:16.515479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a7888-94d6-41b3-bae8-f6588af72f9c None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:16.516085 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a7888-94d6-41b3-bae8-f6588af72f9c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:16.516568 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ef34a-f5e9-404c-9578-025cb4cb8134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:16.516833 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a7888-94d6-41b3-bae8-f6588af72f9c None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:16.517213 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ef34a-f5e9-404c-9578-025cb4cb8134 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:16.517495 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ef34a-f5e9-404c-9578-025cb4cb8134 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:16.517762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ef34a-f5e9-404c-9578-025cb4cb8134 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:16.519232 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 135/267] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:16.524677 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e87ff9-5da6-4e76-9315-6932a02e620d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:16.525266 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e87ff9-5da6-4e76-9315-6932a02e620d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:16.525776 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e87ff9-5da6-4e76-9315-6932a02e620d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:16.547508 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-00e87ff9-5da6-4e76-9315-6932a02e620d None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:16.549048 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-00e87ff9-5da6-4e76-9315-6932a02e620d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:16.551330 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:16.594402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8ef34a-f5e9-404c-9578-025cb4cb8134 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:16.595759 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8ef34a-f5e9-404c-9578-025cb4cb8134 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:16.626870 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 136/268] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:23:16.651029 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e87ff9-5da6-4e76-9315-6932a02e620d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:16.661559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:16.702256 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 133/269] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:23:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:23:16.708068 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ef34a-f5e9-404c-9578-025cb4cb8134 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:16.774615 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e87ff9-5da6-4e76-9315-6932a02e620d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:16.785565 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:16.846196 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ef34a-f5e9-404c-9578-025cb4cb8134 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:16.866434 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc892813-4418-4891-aade-5d6476155159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:16.867092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc892813-4418-4891-aade-5d6476155159 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:16.867372 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc892813-4418-4891-aade-5d6476155159 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:16.867637 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc892813-4418-4891-aade-5d6476155159 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:16.885632 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347fd90-8c0c-40b5-9de3-347e83dc3aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:16.886677 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347fd90-8c0c-40b5-9de3-347e83dc3aff None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:16.887141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347fd90-8c0c-40b5-9de3-347e83dc3aff None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:16.887595 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347fd90-8c0c-40b5-9de3-347e83dc3aff None admin] PATH_INFO: `/v3/projects/3ab93f396edd405c96c6effd6d5edb41` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:16.888930 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347fd90-8c0c-40b5-9de3-347e83dc3aff None admin] RBAC: Authorizing `identity:get_project(project_id=3ab93f396edd405c96c6effd6d5edb41)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:16.891181 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347fd90-8c0c-40b5-9de3-347e83dc3aff None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:16.893050 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 134/270] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:16 2022] GET /identity/v3/projects/3ab93f396edd405c96c6effd6d5edb41 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:16.899484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347fd90-8c0c-40b5-9de3-347e83dc3aff None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:16.912851 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc892813-4418-4891-aade-5d6476155159 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:16.918445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-fc892813-4418-4891-aade-5d6476155159 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:23:16.918445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:16.918445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:16.918445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:16.918445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:16.918445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:16.918445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:16.918445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:16.918445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:16.918445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:16.918445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:16.918445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:16.918445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:16.918445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:16.918445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:16.918445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:16.919362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:16.919362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:16.919362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:16.919362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:16.919362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:16.919362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:16.919362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:16.919362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:16.919362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:16.919362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:16.919362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:16.919362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:16.919362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:16.919362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:16.919362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:16.919362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:16.920185 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:16.920185 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:16.920185 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:16.920185 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:16.920185 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:16.920185 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:16.920185 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:16.920185 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:16.920185 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:16.920185 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:16.920185 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:23:16.920185 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:16.923888 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 137/271] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:16.930578 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc892813-4418-4891-aade-5d6476155159 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:17.003501 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:17.043647 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc892813-4418-4891-aade-5d6476155159 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:17.081122 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12be5776-f7b2-4877-a7e6-ccbb2b5ec394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:17.081763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be5776-f7b2-4877-a7e6-ccbb2b5ec394 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:17.082043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be5776-f7b2-4877-a7e6-ccbb2b5ec394 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:17.082471 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be5776-f7b2-4877-a7e6-ccbb2b5ec394 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:17.083457 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12be5776-f7b2-4877-a7e6-ccbb2b5ec394 None admin] RBAC: Adding query filter params (name=member) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:17.083762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12be5776-f7b2-4877-a7e6-ccbb2b5ec394 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:17.085638 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12be5776-f7b2-4877-a7e6-ccbb2b5ec394 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:17.088806 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347fd90-8c0c-40b5-9de3-347e83dc3aff None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:17.096183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 138/272] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 08:23:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:17.097866 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a17112-e36b-46ad-b331-dce05cc2e547 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:17.098152 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a17112-e36b-46ad-b331-dce05cc2e547 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:17.098479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a17112-e36b-46ad-b331-dce05cc2e547 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:17.100535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 139/273] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:17.105030 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a17112-e36b-46ad-b331-dce05cc2e547 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:17.117862 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:17.126878 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b833c6e-836c-4676-a30f-68e32ce14424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:17.127484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b833c6e-836c-4676-a30f-68e32ce14424 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:17.131456 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b833c6e-836c-4676-a30f-68e32ce14424 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:17.131456 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b833c6e-836c-4676-a30f-68e32ce14424 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:17.131456 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b833c6e-836c-4676-a30f-68e32ce14424 None admin] RBAC: Adding query filter params (role.id=152f4082bb67479ebaeb08c2fa683399, scope.project.id=3ab93f396edd405c96c6effd6d5edb41, user.id=b369df7c929c476ebfc35f907c63d466) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:17.131456 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b833c6e-836c-4676-a30f-68e32ce14424 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:17.131456 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b833c6e-836c-4676-a30f-68e32ce14424 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:17.136482 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 135/274] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:23:16 2022] GET /identity/v3/role_assignments?user.id=b369df7c929c476ebfc35f907c63d466&scope.project.id=3ab93f396edd405c96c6effd6d5edb41&role.id=152f4082bb67479ebaeb08c2fa683399 => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:17.137656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03db08-4eae-4719-b52d-d474bfbf9b9a None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:17.137936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03db08-4eae-4719-b52d-d474bfbf9b9a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:17.138205 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03db08-4eae-4719-b52d-d474bfbf9b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:17.160188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-6e03db08-4eae-4719-b52d-d474bfbf9b9a None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:17.161435 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6e03db08-4eae-4719-b52d-d474bfbf9b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:17.223112 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a17112-e36b-46ad-b331-dce05cc2e547 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:17.236001 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 136/275] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:17.252383 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:17.259765 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03db08-4eae-4719-b52d-d474bfbf9b9a None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:17.265348 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea06a7-4d44-438e-a03e-1f2dee0cc1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:17.266000 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea06a7-4d44-438e-a03e-1f2dee0cc1cb None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:17.266335 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea06a7-4d44-438e-a03e-1f2dee0cc1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:17.266660 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea06a7-4d44-438e-a03e-1f2dee0cc1cb None admin] PATH_INFO: `/v3/users/8d43ed94972541e796422c50ec5b5966` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:17.284109 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaea06a7-4d44-438e-a03e-1f2dee0cc1cb None admin] RBAC: Authorizing `identity:get_user(user_id=8d43ed94972541e796422c50ec5b5966)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:17.285727 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaea06a7-4d44-438e-a03e-1f2dee0cc1cb None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:17.287742 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 140/276] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:23:17 2022] GET /identity/v3/users/8d43ed94972541e796422c50ec5b5966 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:17.293420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea06a7-4d44-438e-a03e-1f2dee0cc1cb None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:17.367278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03db08-4eae-4719-b52d-d474bfbf9b9a None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:17.391776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:17.404286 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8661c-83d5-48bf-87f5-e8185e1015ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:17.406660 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8661c-83d5-48bf-87f5-e8185e1015ed None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:17.406660 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8661c-83d5-48bf-87f5-e8185e1015ed None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:17.406660 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8661c-83d5-48bf-87f5-e8185e1015ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:17.409244 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea06a7-4d44-438e-a03e-1f2dee0cc1cb None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:17.412730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b8661c-83d5-48bf-87f5-e8185e1015ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:17.417100 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b8661c-83d5-48bf-87f5-e8185e1015ed None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:17.419188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 137/277] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:17.425269 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8661c-83d5-48bf-87f5-e8185e1015ed None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:17.449006 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4e54a5-d928-4b61-a781-3efb652edead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:17.449662 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e54a5-d928-4b61-a781-3efb652edead None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:17.449985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e54a5-d928-4b61-a781-3efb652edead None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:17.450264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e54a5-d928-4b61-a781-3efb652edead None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:17.451101 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e54a5-d928-4b61-a781-3efb652edead None admin] RBAC: Adding query filter params (role.id=b0ba6e2bf793402abe2e680c2a7dcb53, user.id=8d43ed94972541e796422c50ec5b5966, scope.system=all) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:17.451460 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e54a5-d928-4b61-a781-3efb652edead None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:17.453127 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e54a5-d928-4b61-a781-3efb652edead None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:17.459638 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 141/278] 10.222.0.33 () {56 vars in 1308 bytes} [Mon Aug 8 08:23:17 2022] GET /identity/v3/role_assignments?user.id=8d43ed94972541e796422c50ec5b5966&scope.system=all&role.id=b0ba6e2bf793402abe2e680c2a7dcb53 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:17.521668 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e72638-95ea-43de-a3ee-f2938b0ade06 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:17.522032 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e72638-95ea-43de-a3ee-f2938b0ade06 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:17.522476 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e72638-95ea-43de-a3ee-f2938b0ade06 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:17.524086 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 142/279] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 08:23:17.530069 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b99136-3ccb-4e45-b86c-7c8cc5f3bc3c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:17.530370 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b99136-3ccb-4e45-b86c-7c8cc5f3bc3c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:17.531145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b99136-3ccb-4e45-b86c-7c8cc5f3bc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:17.536526 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8661c-83d5-48bf-87f5-e8185e1015ed None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:17.552961 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b4b99136-3ccb-4e45-b86c-7c8cc5f3bc3c None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:17.554315 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b4b99136-3ccb-4e45-b86c-7c8cc5f3bc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:17.575431 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae14197b-b61c-427f-8809-6f666c09ed84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:17.576047 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae14197b-b61c-427f-8809-6f666c09ed84 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:17.576319 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae14197b-b61c-427f-8809-6f666c09ed84 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:17.576584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae14197b-b61c-427f-8809-6f666c09ed84 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:17.577265 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae14197b-b61c-427f-8809-6f666c09ed84 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:17.579309 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae14197b-b61c-427f-8809-6f666c09ed84 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:17.607907 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:17.638518 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 143/280] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:23:17.662035 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b99136-3ccb-4e45-b86c-7c8cc5f3bc3c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:17.699824 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 138/281] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 08:23:17 2022] POST /identity/v3/users => generated 293 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:23:17.783579 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b99136-3ccb-4e45-b86c-7c8cc5f3bc3c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:17.823095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022baa97-f2c0-4eee-80a2-36b67fb6446f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:17.823753 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022baa97-f2c0-4eee-80a2-36b67fb6446f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:17.824033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022baa97-f2c0-4eee-80a2-36b67fb6446f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:17.824304 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022baa97-f2c0-4eee-80a2-36b67fb6446f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:17.837622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022baa97-f2c0-4eee-80a2-36b67fb6446f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:17.842763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-022baa97-f2c0-4eee-80a2-36b67fb6446f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:23:17.842763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:17.842763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:17.842763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:17.842763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:17.842763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:17.842763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:17.842763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:17.842763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:17.842763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:17.842763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:17.842763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:17.842763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:17.842763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:17.842763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:17.842763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:17.843769 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:17.843769 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:17.843769 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:17.843769 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:17.843769 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:17.843769 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:17.843769 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:17.843769 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:17.843769 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:17.843769 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:17.843769 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:17.843769 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:17.843769 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:17.843769 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:17.843769 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:17.843769 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:17.844555 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:17.844555 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:17.844555 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:17.844555 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:17.844555 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:17.844555 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:17.844555 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:17.844555 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:17.844555 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:17.844555 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:17.844555 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:23:17.844555 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:17.844555 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 144/282] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:23:17.851001 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae14197b-b61c-427f-8809-6f666c09ed84 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:17.903900 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:17.961716 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae14197b-b61c-427f-8809-6f666c09ed84 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:17.997027 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75614ec-d715-4a7b-b582-024327a1f9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:17.997681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75614ec-d715-4a7b-b582-024327a1f9e0 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:17.997956 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75614ec-d715-4a7b-b582-024327a1f9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:17.998227 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75614ec-d715-4a7b-b582-024327a1f9e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:17.998959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75614ec-d715-4a7b-b582-024327a1f9e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:17.999295 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75614ec-d715-4a7b-b582-024327a1f9e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:18.000913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75614ec-d715-4a7b-b582-024327a1f9e0 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:18.008686 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1788e-2f4d-4fdb-add9-c3589fcf9b11 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:18.008987 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1788e-2f4d-4fdb-add9-c3589fcf9b11 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:18.009289 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1788e-2f4d-4fdb-add9-c3589fcf9b11 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:18.010788 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 145/283] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:18.012873 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 139/284] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 08:23:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:18.015739 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75614ec-d715-4a7b-b582-024327a1f9e0 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:18.016013 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0cea7-39bd-4758-a386-7218964a7c35 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:18.016304 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0cea7-39bd-4758-a386-7218964a7c35 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:18.016586 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0cea7-39bd-4758-a386-7218964a7c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:18.039340 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-1aa0cea7-39bd-4758-a386-7218964a7c35 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:18.040461 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1aa0cea7-39bd-4758-a386-7218964a7c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:18.077181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:18.127458 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 146/285] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:18.144759 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75614ec-d715-4a7b-b582-024327a1f9e0 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:18.150989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa0cea7-39bd-4758-a386-7218964a7c35 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:18.156195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:18.191279 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36659c3f-1084-4d9c-8713-3b6b45d743af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:18.191930 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36659c3f-1084-4d9c-8713-3b6b45d743af None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:18.192198 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36659c3f-1084-4d9c-8713-3b6b45d743af None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:18.192519 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36659c3f-1084-4d9c-8713-3b6b45d743af None admin] PATH_INFO: `/v3/users/a585538b793e490d998d7cf6ffb2d8f8` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:18.193746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36659c3f-1084-4d9c-8713-3b6b45d743af None admin] RBAC: Authorizing `identity:get_user(user_id=a585538b793e490d998d7cf6ffb2d8f8)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:18.195295 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36659c3f-1084-4d9c-8713-3b6b45d743af None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:18.196905 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 140/286] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:23:18 2022] GET /identity/v3/users/a585538b793e490d998d7cf6ffb2d8f8 => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:18.203786 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36659c3f-1084-4d9c-8713-3b6b45d743af None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:18.295741 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa0cea7-39bd-4758-a386-7218964a7c35 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:18.343548 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a681b8-c638-4750-b34b-7343c6381093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:18.344195 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a681b8-c638-4750-b34b-7343c6381093 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:18.344718 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a681b8-c638-4750-b34b-7343c6381093 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:18.344993 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a681b8-c638-4750-b34b-7343c6381093 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:18.356374 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a681b8-c638-4750-b34b-7343c6381093 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:18.362253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-48a681b8-c638-4750-b34b-7343c6381093 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 08:23:18.362253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:18.362253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:18.362253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:18.362253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:18.362253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:18.362253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:18.362253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:18.362253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:18.362253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:18.362253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:18.362253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:18.362253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:18.362253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:18.362253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:18.362253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:18.363131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:18.363131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:18.363131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:18.363131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:18.363131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:18.363131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:18.363131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:18.363131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:18.363131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:18.363131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:18.363131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:18.363131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:18.363131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:18.363131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:18.363131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:18.363131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:18.363913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:18.363913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:18.363913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:18.363913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:18.363913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:18.363913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:18.363913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:18.363913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:18.363913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:18.363913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:18.363913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 08:23:18.363913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:18.363913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 147/287] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:18.369332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a681b8-c638-4750-b34b-7343c6381093 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:18.385799 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36659c3f-1084-4d9c-8713-3b6b45d743af None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:18.416020 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:18.435065 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc8d1b-4fc6-41e6-b054-f63238cd0f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:18.435672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc8d1b-4fc6-41e6-b054-f63238cd0f15 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:18.435993 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc8d1b-4fc6-41e6-b054-f63238cd0f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:18.436267 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc8d1b-4fc6-41e6-b054-f63238cd0f15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:18.510887 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:18.550000 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cc8d1b-4fc6-41e6-b054-f63238cd0f15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:18.552135 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cc8d1b-4fc6-41e6-b054-f63238cd0f15 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:18.554039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a681b8-c638-4750-b34b-7343c6381093 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:18.614538 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f9f6e-790e-4de2-b5c4-773844968acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:18.615162 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f9f6e-790e-4de2-b5c4-773844968acd None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:18.615440 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f9f6e-790e-4de2-b5c4-773844968acd None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:18.615707 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f9f6e-790e-4de2-b5c4-773844968acd None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:18.616384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9f9f6e-790e-4de2-b5c4-773844968acd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:18.616680 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9f9f6e-790e-4de2-b5c4-773844968acd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:18.618086 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9f9f6e-790e-4de2-b5c4-773844968acd None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:18.628063 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 148/288] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 08:23:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:18.633858 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f9f6e-790e-4de2-b5c4-773844968acd None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:18.754984 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 141/289] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:23:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:23:18.762463 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc8d1b-4fc6-41e6-b054-f63238cd0f15 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:18.776863 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:18.801615 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f9f6e-790e-4de2-b5c4-773844968acd None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:18.838550 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc09725-2dc9-4909-8470-4a2bfcb0ab2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:18.839900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc09725-2dc9-4909-8470-4a2bfcb0ab2c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:18.840356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc09725-2dc9-4909-8470-4a2bfcb0ab2c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:18.841157 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc09725-2dc9-4909-8470-4a2bfcb0ab2c None admin] PATH_INFO: `/v3/users/0578f9ae72c54a05adcf7de0ae06ca36` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:18.842387 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc09725-2dc9-4909-8470-4a2bfcb0ab2c None admin] RBAC: Authorizing `identity:get_user(user_id=0578f9ae72c54a05adcf7de0ae06ca36)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:18.844668 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc09725-2dc9-4909-8470-4a2bfcb0ab2c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:18.846900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 149/290] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:23:18 2022] GET /identity/v3/users/0578f9ae72c54a05adcf7de0ae06ca36 => generated 297 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:18.855155 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc09725-2dc9-4909-8470-4a2bfcb0ab2c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:18.889534 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:18.921342 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc8d1b-4fc6-41e6-b054-f63238cd0f15 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:18.982631 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b246e-c893-4f9a-9cb0-437a9c5b289d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:18.983903 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b246e-c893-4f9a-9cb0-437a9c5b289d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:18.984364 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b246e-c893-4f9a-9cb0-437a9c5b289d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:18.984804 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b246e-c893-4f9a-9cb0-437a9c5b289d None admin] PATH_INFO: `/v3/projects/3ab93f396edd405c96c6effd6d5edb41` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:18.986223 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b246e-c893-4f9a-9cb0-437a9c5b289d None admin] RBAC: Authorizing `identity:get_project(project_id=3ab93f396edd405c96c6effd6d5edb41)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:18.987800 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b246e-c893-4f9a-9cb0-437a9c5b289d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:18.989903 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 142/291] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:18 2022] GET /identity/v3/projects/3ab93f396edd405c96c6effd6d5edb41 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:18.997562 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b246e-c893-4f9a-9cb0-437a9c5b289d None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:19.028200 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc09725-2dc9-4909-8470-4a2bfcb0ab2c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:19.078339 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebda82a-5562-4120-9780-338333316c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:19.079267 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebda82a-5562-4120-9780-338333316c7a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:19.079699 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebda82a-5562-4120-9780-338333316c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:19.080150 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebda82a-5562-4120-9780-338333316c7a None admin] PATH_INFO: `/v3/projects/3ab93f396edd405c96c6effd6d5edb41` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:19.083118 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebda82a-5562-4120-9780-338333316c7a None admin] RBAC: Authorizing `identity:get_project(project_id=3ab93f396edd405c96c6effd6d5edb41)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:19.084723 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebda82a-5562-4120-9780-338333316c7a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:19.088863 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 150/292] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:18 2022] GET /identity/v3/projects/3ab93f396edd405c96c6effd6d5edb41 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:19.095143 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebda82a-5562-4120-9780-338333316c7a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:19.147976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:19.186017 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b246e-c893-4f9a-9cb0-437a9c5b289d None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:19.236388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac89977-7f60-4228-a01d-aef4f81b6e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:19.237841 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac89977-7f60-4228-a01d-aef4f81b6e2e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:19.238258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac89977-7f60-4228-a01d-aef4f81b6e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:19.238740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac89977-7f60-4228-a01d-aef4f81b6e2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:19.239605 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac89977-7f60-4228-a01d-aef4f81b6e2e None admin] RBAC: Adding query filter params (role.id=b0ba6e2bf793402abe2e680c2a7dcb53, scope.project.id=3ab93f396edd405c96c6effd6d5edb41, user.id=a585538b793e490d998d7cf6ffb2d8f8) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:19.240074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac89977-7f60-4228-a01d-aef4f81b6e2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:19.241402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebda82a-5562-4120-9780-338333316c7a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:19.241823 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac89977-7f60-4228-a01d-aef4f81b6e2e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:19.256274 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 143/293] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:23:18 2022] GET /identity/v3/role_assignments?user.id=a585538b793e490d998d7cf6ffb2d8f8&scope.project.id=3ab93f396edd405c96c6effd6d5edb41&role.id=b0ba6e2bf793402abe2e680c2a7dcb53 => generated 608 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:19.316035 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1888225c-edd9-48ff-9458-3a140c39ac54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:19.316719 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888225c-edd9-48ff-9458-3a140c39ac54 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:19.317002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888225c-edd9-48ff-9458-3a140c39ac54 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:19.317305 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888225c-edd9-48ff-9458-3a140c39ac54 None admin] PATH_INFO: `/v3/projects/3ab93f396edd405c96c6effd6d5edb41/users/0578f9ae72c54a05adcf7de0ae06ca36/roles/ab682c3121c04a5fb600dc47750d013c` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:19.322695 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1888225c-edd9-48ff-9458-3a140c39ac54 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ab93f396edd405c96c6effd6d5edb41, user_id=0578f9ae72c54a05adcf7de0ae06ca36, role_id=ab682c3121c04a5fb600dc47750d013c)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:19.324157 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1888225c-edd9-48ff-9458-3a140c39ac54 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:19.395287 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 151/294] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 08:23:19 2022] PUT /identity/v3/projects/3ab93f396edd405c96c6effd6d5edb41/users/0578f9ae72c54a05adcf7de0ae06ca36/roles/ab682c3121c04a5fb600dc47750d013c => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:23:19.752416 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88752353-414f-42bf-9879-7d0b58d2c8b2 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:19.752748 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88752353-414f-42bf-9879-7d0b58d2c8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:19.752923 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88752353-414f-42bf-9879-7d0b58d2c8b2 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:19.754486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 144/295] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:19.759904 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b41cdd-3c9c-4333-ac7a-e697f125b305 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:19.760192 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b41cdd-3c9c-4333-ac7a-e697f125b305 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:19.760463 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b41cdd-3c9c-4333-ac7a-e697f125b305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:19.778889 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b6b41cdd-3c9c-4333-ac7a-e697f125b305 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:19.780074 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b6b41cdd-3c9c-4333-ac7a-e697f125b305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:19.852141 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 152/296] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:19.873997 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88752353-414f-42bf-9879-7d0b58d2c8b2 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:19.885719 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:19.980810 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88752353-414f-42bf-9879-7d0b58d2c8b2 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:20.016719 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6d486-cadf-47c8-87d4-d53729ab6710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:20.017304 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6d486-cadf-47c8-87d4-d53729ab6710 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:20.017586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6d486-cadf-47c8-87d4-d53729ab6710 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:20.017858 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6d486-cadf-47c8-87d4-d53729ab6710 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:20.028267 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea6d486-cadf-47c8-87d4-d53729ab6710 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:20.033670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-3ea6d486-cadf-47c8-87d4-d53729ab6710 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:23:20.033670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:20.033670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:20.033670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:20.033670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:20.033670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:20.033670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:20.033670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:20.033670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:20.033670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:20.033670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:20.033670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:20.033670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:20.033670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:20.033670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:20.033670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:20.034557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:20.034557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:20.034557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:20.034557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:20.034557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:20.034557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:20.034557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:20.034557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:20.034557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:20.034557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:20.034557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:20.034557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:20.034557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:20.034557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:20.034557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:20.034557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:20.035356 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:20.035356 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:20.035356 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:20.035356 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:20.035356 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:20.035356 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:20.035356 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:20.035356 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:20.035356 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:20.035356 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:20.035356 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:23:20.035356 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:20.035356 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 145/297] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:20.042492 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b41cdd-3c9c-4333-ac7a-e697f125b305 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:20.058187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:20.148152 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b41cdd-3c9c-4333-ac7a-e697f125b305 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:20.184054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2ac94-a2d9-496b-bfe2-59327807e15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:20.184672 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2ac94-a2d9-496b-bfe2-59327807e15b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:20.184954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2ac94-a2d9-496b-bfe2-59327807e15b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:20.185252 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2ac94-a2d9-496b-bfe2-59327807e15b None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:20.185927 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba2ac94-a2d9-496b-bfe2-59327807e15b None admin] RBAC: Adding query filter params (name=member) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:20.186227 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba2ac94-a2d9-496b-bfe2-59327807e15b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:20.187679 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba2ac94-a2d9-496b-bfe2-59327807e15b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:20.196830 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 153/298] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 08:23:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:20.202125 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6d486-cadf-47c8-87d4-d53729ab6710 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:20.273850 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:20.274783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ada88-6e06-4abf-a0a0-420f75430098 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:20.275293 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ada88-6e06-4abf-a0a0-420f75430098 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:20.275767 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ada88-6e06-4abf-a0a0-420f75430098 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:20.277693 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 154/299] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 08:23:20.283041 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506bba3-365d-4e4e-982c-ecc7bebbfdc8 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:20.283328 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506bba3-365d-4e4e-982c-ecc7bebbfdc8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:20.288722 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506bba3-365d-4e4e-982c-ecc7bebbfdc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:20.302294 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-2506bba3-365d-4e4e-982c-ecc7bebbfdc8 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:20.303584 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2506bba3-365d-4e4e-982c-ecc7bebbfdc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:20.310879 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6d486-cadf-47c8-87d4-d53729ab6710 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:20.346169 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d5d4a-0eb0-46cc-a170-7494a12ca54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:20.346793 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d5d4a-0eb0-46cc-a170-7494a12ca54f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:20.347075 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d5d4a-0eb0-46cc-a170-7494a12ca54f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:20.347346 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d5d4a-0eb0-46cc-a170-7494a12ca54f None admin] PATH_INFO: `/v3/users/8d43ed94972541e796422c50ec5b5966` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:20.348470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d5d4a-0eb0-46cc-a170-7494a12ca54f None admin] RBAC: Authorizing `identity:get_user(user_id=8d43ed94972541e796422c50ec5b5966)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:20.349933 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d5d4a-0eb0-46cc-a170-7494a12ca54f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:20.351811 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 146/300] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:23:20 2022] GET /identity/v3/users/8d43ed94972541e796422c50ec5b5966 => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:20.352924 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f15bc1-804d-4974-b3ba-5925c3456efd None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:20.353391 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f15bc1-804d-4974-b3ba-5925c3456efd None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:20.353495 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f15bc1-804d-4974-b3ba-5925c3456efd None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:20.354890 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 147/301] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:20.358526 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f15bc1-804d-4974-b3ba-5925c3456efd None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:20.379467 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 155/302] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:23:20.381449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de811357-0259-4106-85f6-7b02b9458252 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:20.381877 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de811357-0259-4106-85f6-7b02b9458252 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:20.382290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de811357-0259-4106-85f6-7b02b9458252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:20.401539 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:20.411216 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-de811357-0259-4106-85f6-7b02b9458252 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:20.412550 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-de811357-0259-4106-85f6-7b02b9458252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:20.472391 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f15bc1-804d-4974-b3ba-5925c3456efd None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:20.482512 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 156/303] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:20.483146 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de811357-0259-4106-85f6-7b02b9458252 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:20.519738 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68843e-cd77-45c1-9d7e-e5687d410b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:20.520392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68843e-cd77-45c1-9d7e-e5687d410b3d None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:20.520673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68843e-cd77-45c1-9d7e-e5687d410b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:20.520950 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68843e-cd77-45c1-9d7e-e5687d410b3d None admin] PATH_INFO: `/v3/system/users/8d43ed94972541e796422c50ec5b5966/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:20.522479 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c68843e-cd77-45c1-9d7e-e5687d410b3d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8d43ed94972541e796422c50ec5b5966, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:20.523914 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c68843e-cd77-45c1-9d7e-e5687d410b3d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:20.528672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:20.583449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de811357-0259-4106-85f6-7b02b9458252 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:20.616715 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 148/304] 10.222.0.33 () {58 vars in 1334 bytes} [Mon Aug 8 08:23:20 2022] PUT /identity/v3/system/users/8d43ed94972541e796422c50ec5b5966/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:23:20.617568 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68843e-cd77-45c1-9d7e-e5687d410b3d None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:20.627559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceee1c6-1726-4dbc-a9f6-541032afda4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:20.628226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceee1c6-1726-4dbc-a9f6-541032afda4f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:20.628508 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceee1c6-1726-4dbc-a9f6-541032afda4f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:20.628774 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceee1c6-1726-4dbc-a9f6-541032afda4f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:20.634334 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceee1c6-1726-4dbc-a9f6-541032afda4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:20.634334 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceee1c6-1726-4dbc-a9f6-541032afda4f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:20.634334 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 157/305] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:20.640972 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceee1c6-1726-4dbc-a9f6-541032afda4f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:20.646629 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:20.754982 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68843e-cd77-45c1-9d7e-e5687d410b3d None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:20.788968 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceee1c6-1726-4dbc-a9f6-541032afda4f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:20.796751 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3da89b-108d-4ee5-b121-28db2a23a4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:20.797560 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3da89b-108d-4ee5-b121-28db2a23a4b2 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:20.797952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3da89b-108d-4ee5-b121-28db2a23a4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:20.798225 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3da89b-108d-4ee5-b121-28db2a23a4b2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:20.813882 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3da89b-108d-4ee5-b121-28db2a23a4b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:20.821313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-fb3da89b-108d-4ee5-b121-28db2a23a4b2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 08:23:20.821313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:20.821313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:20.821313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:20.821313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:20.821313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:20.821313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:20.821313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:20.821313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:20.821313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:20.821313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:20.821313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:20.821313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:20.821313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:20.821313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:20.821313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:20.822283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:20.822283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:20.822283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:20.822283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:20.822283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:20.822283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:20.822283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:20.822283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:20.822283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:20.822283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:20.822283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:20.822283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:20.822283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:20.822283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:20.822283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:20.822283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:20.823144 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:20.823144 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:20.823144 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:20.823144 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:20.823144 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:20.823144 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:20.823144 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:20.823144 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:20.823144 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:20.823144 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:20.823144 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 08:23:20.823144 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:20.823144 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 149/306] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:20.828378 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3da89b-108d-4ee5-b121-28db2a23a4b2 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:20.831804 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4648f-9464-4ce5-81f6-93b0ff6fbc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:20.832600 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4648f-9464-4ce5-81f6-93b0ff6fbc9f None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:20.833071 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4648f-9464-4ce5-81f6-93b0ff6fbc9f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:20.833396 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4648f-9464-4ce5-81f6-93b0ff6fbc9f None admin] PATH_INFO: `/v3/groups` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:20.834025 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d4648f-9464-4ce5-81f6-93b0ff6fbc9f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:20.835449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d4648f-9464-4ce5-81f6-93b0ff6fbc9f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:20.892321 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:20.901465 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 158/307] 10.222.0.33 () {60 vars in 1134 bytes} [Mon Aug 8 08:23:20 2022] POST /identity/v3/groups => generated 229 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:23:20.947756 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3da89b-108d-4ee5-b121-28db2a23a4b2 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:20.988817 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45bb448-2cca-46e1-9735-53843ce0f124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:20.989870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bb448-2cca-46e1-9735-53843ce0f124 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:20.990399 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bb448-2cca-46e1-9735-53843ce0f124 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:20.990905 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bb448-2cca-46e1-9735-53843ce0f124 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:20.991802 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45bb448-2cca-46e1-9735-53843ce0f124 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:20.992331 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45bb448-2cca-46e1-9735-53843ce0f124 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:20.994020 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45bb448-2cca-46e1-9735-53843ce0f124 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:21.005095 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 150/308] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 08:23:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:21.011435 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4648f-9464-4ce5-81f6-93b0ff6fbc9f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:21.083006 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:21.122946 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4648f-9464-4ce5-81f6-93b0ff6fbc9f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:21.159961 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7c2cf-fc4b-4c51-9bdf-afc2caf9fd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:21.161003 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7c2cf-fc4b-4c51-9bdf-afc2caf9fd4b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:21.161496 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7c2cf-fc4b-4c51-9bdf-afc2caf9fd4b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:21.161977 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7c2cf-fc4b-4c51-9bdf-afc2caf9fd4b None admin] PATH_INFO: `/v3/users/f4ac3dc48fda4923b9e96bf8ce95537d` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:21.177292 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b7c2cf-fc4b-4c51-9bdf-afc2caf9fd4b None admin] RBAC: Authorizing `identity:get_user(user_id=f4ac3dc48fda4923b9e96bf8ce95537d)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:21.179090 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b7c2cf-fc4b-4c51-9bdf-afc2caf9fd4b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:21.181299 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 159/309] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:23:21 2022] GET /identity/v3/users/f4ac3dc48fda4923b9e96bf8ce95537d => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:21.188139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45bb448-2cca-46e1-9735-53843ce0f124 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:21.292874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45bb448-2cca-46e1-9735-53843ce0f124 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:21.324637 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:21.329659 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca81c06-a4f9-4a61-ad13-1afba50c37f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:21.330849 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca81c06-a4f9-4a61-ad13-1afba50c37f1 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:21.331336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca81c06-a4f9-4a61-ad13-1afba50c37f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:21.331851 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca81c06-a4f9-4a61-ad13-1afba50c37f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:21.332772 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca81c06-a4f9-4a61-ad13-1afba50c37f1 None admin] RBAC: Adding query filter params (role.id=ab682c3121c04a5fb600dc47750d013c, user.id=f4ac3dc48fda4923b9e96bf8ce95537d, scope.system=all) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:21.345228 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca81c06-a4f9-4a61-ad13-1afba50c37f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:21.345228 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca81c06-a4f9-4a61-ad13-1afba50c37f1 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:21.345228 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 151/310] 10.222.0.33 () {56 vars in 1308 bytes} [Mon Aug 8 08:23:21 2022] GET /identity/v3/role_assignments?user.id=f4ac3dc48fda4923b9e96bf8ce95537d&scope.system=all&role.id=ab682c3121c04a5fb600dc47750d013c => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:21.593969 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf3b84-7145-4cb4-bc17-e0789c5b0124 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:21.594474 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf3b84-7145-4cb4-bc17-e0789c5b0124 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:21.594578 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf3b84-7145-4cb4-bc17-e0789c5b0124 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:21.596059 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 160/311] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:21.601070 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c5ebc-d23f-441b-96c8-f76ff2d86193 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:21.601387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c5ebc-d23f-441b-96c8-f76ff2d86193 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:21.601664 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c5ebc-d23f-441b-96c8-f76ff2d86193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:21.625783 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-d18c5ebc-d23f-441b-96c8-f76ff2d86193 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:21.626967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d18c5ebc-d23f-441b-96c8-f76ff2d86193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:21.694484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 152/312] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:21.717864 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf3b84-7145-4cb4-bc17-e0789c5b0124 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:21.827740 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf3b84-7145-4cb4-bc17-e0789c5b0124 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:21.853762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:21.863641 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c40c26-9204-442b-ba0c-aa9c24b15ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:21.864296 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c40c26-9204-442b-ba0c-aa9c24b15ebe None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:21.864577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c40c26-9204-442b-ba0c-aa9c24b15ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:21.864849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c40c26-9204-442b-ba0c-aa9c24b15ebe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:21.876241 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c40c26-9204-442b-ba0c-aa9c24b15ebe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:21.881028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-d2c40c26-9204-442b-ba0c-aa9c24b15ebe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 08:23:21.881028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:21.881028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:21.881028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:21.881028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:21.881028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:21.881028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:21.881028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:21.881028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:21.881028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:21.881028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:21.881028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:21.881028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:21.881028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:21.881028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:21.881028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:21.881913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:21.881913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:21.881913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:21.881913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:21.881913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:21.881913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:21.881913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:21.881913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:21.881913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:21.881913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:21.881913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:21.881913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:21.881913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:21.881913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:21.881913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:21.881913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:21.882695 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:21.882695 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:21.882695 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:21.882695 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:21.882695 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:21.882695 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:21.882695 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:21.882695 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:21.882695 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:21.882695 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:21.882695 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 08:23:21.882695 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:21.882695 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 161/313] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:21.888462 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c5ebc-d23f-441b-96c8-f76ff2d86193 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:21.987680 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c5ebc-d23f-441b-96c8-f76ff2d86193 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:22.007856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:22.025921 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a98fc4-141d-4c87-8049-c8ceb98da97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:22.026846 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a98fc4-141d-4c87-8049-c8ceb98da97a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:22.027317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a98fc4-141d-4c87-8049-c8ceb98da97a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:22.027798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a98fc4-141d-4c87-8049-c8ceb98da97a None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:22.028670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a98fc4-141d-4c87-8049-c8ceb98da97a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:22.029181 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a98fc4-141d-4c87-8049-c8ceb98da97a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:22.031277 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a98fc4-141d-4c87-8049-c8ceb98da97a None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:22.040261 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 153/314] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 08:23:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:22.045589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c40c26-9204-442b-ba0c-aa9c24b15ebe None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:22.162163 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c40c26-9204-442b-ba0c-aa9c24b15ebe None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:22.197946 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7061c7-b857-49c1-a652-1f2b53852c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:22.198591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7061c7-b857-49c1-a652-1f2b53852c0b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:22.198871 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7061c7-b857-49c1-a652-1f2b53852c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:22.199141 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7061c7-b857-49c1-a652-1f2b53852c0b None admin] PATH_INFO: `/v3/users/0578f9ae72c54a05adcf7de0ae06ca36` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:22.200276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7061c7-b857-49c1-a652-1f2b53852c0b None admin] RBAC: Authorizing `identity:get_user(user_id=0578f9ae72c54a05adcf7de0ae06ca36)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:22.201757 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7061c7-b857-49c1-a652-1f2b53852c0b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:22.204102 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 162/315] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:23:22 2022] GET /identity/v3/users/0578f9ae72c54a05adcf7de0ae06ca36 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:22.209720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a98fc4-141d-4c87-8049-c8ceb98da97a None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:22.311646 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a98fc4-141d-4c87-8049-c8ceb98da97a None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:22.349376 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae51e9-1379-4c5d-ad13-22f7209e8b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:22.349988 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae51e9-1379-4c5d-ad13-22f7209e8b19 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:22.350264 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae51e9-1379-4c5d-ad13-22f7209e8b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:22.350532 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae51e9-1379-4c5d-ad13-22f7209e8b19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:22.397554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:22.420995 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ae51e9-1379-4c5d-ad13-22f7209e8b19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:22.422211 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ae51e9-1379-4c5d-ad13-22f7209e8b19 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:22.527560 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 154/316] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:23:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:23:22.528496 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7061c7-b857-49c1-a652-1f2b53852c0b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:22.587901 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:22.646606 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7061c7-b857-49c1-a652-1f2b53852c0b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:22.685208 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a4af2-e834-4a31-b6a8-434159e865d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:22.685768 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a4af2-e834-4a31-b6a8-434159e865d9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:22.686043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a4af2-e834-4a31-b6a8-434159e865d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:22.686310 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a4af2-e834-4a31-b6a8-434159e865d9 None admin] PATH_INFO: `/v3/projects/3ab93f396edd405c96c6effd6d5edb41` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:22.687461 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a4af2-e834-4a31-b6a8-434159e865d9 None admin] RBAC: Authorizing `identity:get_project(project_id=3ab93f396edd405c96c6effd6d5edb41)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:22.688902 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a4af2-e834-4a31-b6a8-434159e865d9 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:22.690890 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 163/317] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:22 2022] GET /identity/v3/projects/3ab93f396edd405c96c6effd6d5edb41 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:22.696523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae51e9-1379-4c5d-ad13-22f7209e8b19 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:22.747304 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad52ed4-cfa9-40c5-bd62-f508e6e2cd14 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:22.747648 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad52ed4-cfa9-40c5-bd62-f508e6e2cd14 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:22.747824 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad52ed4-cfa9-40c5-bd62-f508e6e2cd14 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:22.749167 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 164/318] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 08:23:22.756486 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628f1f6-b1da-4712-a9ad-c69c69205070 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:22.756874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628f1f6-b1da-4712-a9ad-c69c69205070 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:22.757582 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628f1f6-b1da-4712-a9ad-c69c69205070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:22.780856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8628f1f6-b1da-4712-a9ad-c69c69205070 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:22.782449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8628f1f6-b1da-4712-a9ad-c69c69205070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:22.789509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:22.808392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae51e9-1379-4c5d-ad13-22f7209e8b19 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:22.844718 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fa199-83f7-4159-94de-3a830db4be05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:22.845505 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fa199-83f7-4159-94de-3a830db4be05 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:22.845880 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fa199-83f7-4159-94de-3a830db4be05 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:22.846250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fa199-83f7-4159-94de-3a830db4be05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:22.847507 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fa199-83f7-4159-94de-3a830db4be05 None admin] RBAC: Adding query filter params (role.id=ab682c3121c04a5fb600dc47750d013c, scope.project.id=3ab93f396edd405c96c6effd6d5edb41, user.id=0578f9ae72c54a05adcf7de0ae06ca36) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:22.847917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fa199-83f7-4159-94de-3a830db4be05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:22.849894 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fa199-83f7-4159-94de-3a830db4be05 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:22.857953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 155/319] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:23:22 2022] GET /identity/v3/role_assignments?user.id=0578f9ae72c54a05adcf7de0ae06ca36&scope.project.id=3ab93f396edd405c96c6effd6d5edb41&role.id=ab682c3121c04a5fb600dc47750d013c => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:22.861207 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 165/320] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:22.890504 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fa199-83f7-4159-94de-3a830db4be05 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:23.010318 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fa199-83f7-4159-94de-3a830db4be05 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:23.048494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57acb7b7-5b6a-4333-90fd-84c136c00cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:23.049212 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acb7b7-5b6a-4333-90fd-84c136c00cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:23.049494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acb7b7-5b6a-4333-90fd-84c136c00cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:23.049764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acb7b7-5b6a-4333-90fd-84c136c00cb0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:23.055347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b78a9-b668-4974-afda-19422fdf6ec2 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:23.055922 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b78a9-b668-4974-afda-19422fdf6ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:23.056392 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b78a9-b668-4974-afda-19422fdf6ec2 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:23.057989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 166/321] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 08:23:23.061339 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57acb7b7-5b6a-4333-90fd-84c136c00cb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:23.063687 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493aa66b-656f-4784-be66-e261b5386a87 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:23.064184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493aa66b-656f-4784-be66-e261b5386a87 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:23.064649 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493aa66b-656f-4784-be66-e261b5386a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:23.066385 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-57acb7b7-5b6a-4333-90fd-84c136c00cb0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:23:23.066385 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:23.066385 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:23.066385 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:23.066385 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:23.066385 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:23.066385 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:23.066385 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:23.066385 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:23.066385 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:23.066385 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:23.066385 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:23.066385 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:23.066385 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:23.066385 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:23.066385 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:23.067318 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:23.067318 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:23.067318 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:23.067318 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:23.067318 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:23.067318 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:23.067318 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:23.067318 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:23.067318 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:23.067318 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:23.067318 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:23.067318 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:23.067318 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:23.067318 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:23.067318 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:23.067318 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:23.068134 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:23.068134 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:23.068134 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:23.068134 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:23.068134 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:23.068134 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:23.068134 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:23.068134 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:23.068134 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:23.068134 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:23.068134 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:23:23.068134 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:23.068134 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 156/322] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:23.075388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57acb7b7-5b6a-4333-90fd-84c136c00cb0 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:23.079433 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:23.089344 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-493aa66b-656f-4784-be66-e261b5386a87 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:23.090801 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-493aa66b-656f-4784-be66-e261b5386a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:23.153055 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:23.181110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 167/323] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:23:23.203301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57acb7b7-5b6a-4333-90fd-84c136c00cb0 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:23.206988 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493aa66b-656f-4784-be66-e261b5386a87 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:23.240626 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91accecb-d83f-459f-902e-bb6f6742f4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:23.241268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91accecb-d83f-459f-902e-bb6f6742f4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:23.241545 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91accecb-d83f-459f-902e-bb6f6742f4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:23.241814 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91accecb-d83f-459f-902e-bb6f6742f4f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:23.242475 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91accecb-d83f-459f-902e-bb6f6742f4f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:23.242776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91accecb-d83f-459f-902e-bb6f6742f4f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:23.244143 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91accecb-d83f-459f-902e-bb6f6742f4f2 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:23.254371 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 157/324] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 08:23:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:23.259741 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91accecb-d83f-459f-902e-bb6f6742f4f2 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:23.321666 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493aa66b-656f-4784-be66-e261b5386a87 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:23.323406 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:23.360762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d03e7-924f-4864-9ed4-b3a7e883ed27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:23.361585 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d03e7-924f-4864-9ed4-b3a7e883ed27 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:23.361876 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d03e7-924f-4864-9ed4-b3a7e883ed27 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:23.362153 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d03e7-924f-4864-9ed4-b3a7e883ed27 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:23.363093 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91accecb-d83f-459f-902e-bb6f6742f4f2 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:23.363258 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d03e7-924f-4864-9ed4-b3a7e883ed27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:23.364703 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d03e7-924f-4864-9ed4-b3a7e883ed27 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:23.370913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 168/325] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:23.371853 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d03e7-924f-4864-9ed4-b3a7e883ed27 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:23.399424 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08000b77-3b54-4653-a722-82ddf620d844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:23.400059 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08000b77-3b54-4653-a722-82ddf620d844 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:23.400341 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08000b77-3b54-4653-a722-82ddf620d844 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:23.400616 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08000b77-3b54-4653-a722-82ddf620d844 None admin] PATH_INFO: `/v3/users/8d43ed94972541e796422c50ec5b5966` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:23.401749 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08000b77-3b54-4653-a722-82ddf620d844 None admin] RBAC: Authorizing `identity:get_user(user_id=8d43ed94972541e796422c50ec5b5966)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:23.403186 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08000b77-3b54-4653-a722-82ddf620d844 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:23.404918 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 158/326] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:23:23 2022] GET /identity/v3/users/8d43ed94972541e796422c50ec5b5966 => generated 293 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:23.412048 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08000b77-3b54-4653-a722-82ddf620d844 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:23.431654 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:23.487552 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d03e7-924f-4864-9ed4-b3a7e883ed27 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:23.519093 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08000b77-3b54-4653-a722-82ddf620d844 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:23.520623 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c63d0f-eb62-47ec-9a1c-4f0628d530ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:23.521250 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c63d0f-eb62-47ec-9a1c-4f0628d530ae None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:23.521528 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c63d0f-eb62-47ec-9a1c-4f0628d530ae None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:23.521795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c63d0f-eb62-47ec-9a1c-4f0628d530ae None admin] PATH_INFO: `/v3/groups` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:23.522408 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c63d0f-eb62-47ec-9a1c-4f0628d530ae None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:23.523821 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c63d0f-eb62-47ec-9a1c-4f0628d530ae None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:23.548536 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:23.553225 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae496208-0a0a-48ce-bbc2-3f948b7fb10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:23.553826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae496208-0a0a-48ce-bbc2-3f948b7fb10d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:23.554104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae496208-0a0a-48ce-bbc2-3f948b7fb10d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:23.554374 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae496208-0a0a-48ce-bbc2-3f948b7fb10d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:23.555095 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae496208-0a0a-48ce-bbc2-3f948b7fb10d None admin] RBAC: Adding query filter params (role.id=b0ba6e2bf793402abe2e680c2a7dcb53, user.id=8d43ed94972541e796422c50ec5b5966, scope.system=all) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:23.555402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae496208-0a0a-48ce-bbc2-3f948b7fb10d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:23.556810 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae496208-0a0a-48ce-bbc2-3f948b7fb10d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:23.568179 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 159/327] 10.222.0.33 () {56 vars in 1308 bytes} [Mon Aug 8 08:23:23 2022] GET /identity/v3/role_assignments?user.id=8d43ed94972541e796422c50ec5b5966&scope.system=all&role.id=b0ba6e2bf793402abe2e680c2a7dcb53 => generated 510 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:23.569343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8028b5-cf6a-4cf6-b658-f39144924c21 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:23.569626 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8028b5-cf6a-4cf6-b658-f39144924c21 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:23.569942 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8028b5-cf6a-4cf6-b658-f39144924c21 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:23.571083 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 160/328] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:23.581952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f16c1-d9e2-430f-88a6-411add8d4dba None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:23.581952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f16c1-d9e2-430f-88a6-411add8d4dba None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:23.581952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f16c1-d9e2-430f-88a6-411add8d4dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:23.597131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-630f16c1-d9e2-430f-88a6-411add8d4dba None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:23.598377 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-630f16c1-d9e2-430f-88a6-411add8d4dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:23.605852 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 169/329] 10.222.0.33 () {60 vars in 1134 bytes} [Mon Aug 8 08:23:23 2022] POST /identity/v3/groups => generated 226 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:23:23.678059 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 161/330] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:23:23.706198 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c63d0f-eb62-47ec-9a1c-4f0628d530ae None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:23.819113 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c63d0f-eb62-47ec-9a1c-4f0628d530ae None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:23.826230 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:23.855084 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bee216-f87d-4598-8d94-5534b77ca570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:23.855651 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bee216-f87d-4598-8d94-5534b77ca570 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:23.855932 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bee216-f87d-4598-8d94-5534b77ca570 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:23.856201 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bee216-f87d-4598-8d94-5534b77ca570 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:23.869351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bee216-f87d-4598-8d94-5534b77ca570 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:23.874642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-c1bee216-f87d-4598-8d94-5534b77ca570 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 08:23:23.874642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:23.874642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:23.874642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:23.874642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:23.874642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:23.874642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:23.874642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:23.874642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:23.874642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:23.874642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:23.874642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:23.874642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:23.874642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:23.874642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:23.874642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:23.875723 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:23.875723 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:23.875723 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:23.875723 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:23.875723 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:23.875723 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:23.875723 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:23.875723 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:23.875723 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:23.875723 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:23.875723 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:23.875723 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:23.875723 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:23.875723 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:23.875723 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:23.875723 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:23.876692 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:23.876692 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:23.876692 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:23.876692 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:23.876692 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:23.876692 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:23.876692 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:23.876692 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:23.876692 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:23.876692 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:23.876692 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 08:23:23.876692 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:23.879495 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 170/331] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:23.885040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630f16c1-d9e2-430f-88a6-411add8d4dba None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:23.993320 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630f16c1-d9e2-430f-88a6-411add8d4dba None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:24.015796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:24.030239 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325aaeb5-2aeb-42da-82c4-cc257715a1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:24.031442 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325aaeb5-2aeb-42da-82c4-cc257715a1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:24.032676 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325aaeb5-2aeb-42da-82c4-cc257715a1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:24.032676 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325aaeb5-2aeb-42da-82c4-cc257715a1f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:24.032817 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325aaeb5-2aeb-42da-82c4-cc257715a1f6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:24.033016 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325aaeb5-2aeb-42da-82c4-cc257715a1f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:24.034439 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325aaeb5-2aeb-42da-82c4-cc257715a1f6 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:24.046965 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 162/332] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 08:23:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:24.051181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bee216-f87d-4598-8d94-5534b77ca570 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:24.163205 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bee216-f87d-4598-8d94-5534b77ca570 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:24.198539 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba28cb2-252d-475d-8771-87f97197ec06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:24.199181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba28cb2-252d-475d-8771-87f97197ec06 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:24.199459 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba28cb2-252d-475d-8771-87f97197ec06 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:24.199729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba28cb2-252d-475d-8771-87f97197ec06 None admin] PATH_INFO: `/v3/users/f4ac3dc48fda4923b9e96bf8ce95537d` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:24.200882 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba28cb2-252d-475d-8771-87f97197ec06 None admin] RBAC: Authorizing `identity:get_user(user_id=f4ac3dc48fda4923b9e96bf8ce95537d)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:24.202511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba28cb2-252d-475d-8771-87f97197ec06 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:24.204302 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 171/333] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:23:24 2022] GET /identity/v3/users/f4ac3dc48fda4923b9e96bf8ce95537d => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:24.210022 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325aaeb5-2aeb-42da-82c4-cc257715a1f6 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:24.325761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325aaeb5-2aeb-42da-82c4-cc257715a1f6 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:24.367226 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f81be7-5bfd-4955-b5ba-e14cf0a367b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:24.368255 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f81be7-5bfd-4955-b5ba-e14cf0a367b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:24.368737 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f81be7-5bfd-4955-b5ba-e14cf0a367b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:24.369361 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f81be7-5bfd-4955-b5ba-e14cf0a367b4 None admin] PATH_INFO: `/v3/system/users/f4ac3dc48fda4923b9e96bf8ce95537d/roles/ab682c3121c04a5fb600dc47750d013c` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:24.371016 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f81be7-5bfd-4955-b5ba-e14cf0a367b4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f4ac3dc48fda4923b9e96bf8ce95537d, role_id=ab682c3121c04a5fb600dc47750d013c)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:24.372604 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f81be7-5bfd-4955-b5ba-e14cf0a367b4 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:24.451386 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 163/334] 10.222.0.33 () {58 vars in 1334 bytes} [Mon Aug 8 08:23:24 2022] PUT /identity/v3/system/users/f4ac3dc48fda4923b9e96bf8ce95537d/roles/ab682c3121c04a5fb600dc47750d013c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:23:26.146649 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f4a23-a6ee-4cc9-ac60-e4d598bf1144 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:26.147173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f4a23-a6ee-4cc9-ac60-e4d598bf1144 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:26.147173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f4a23-a6ee-4cc9-ac60-e4d598bf1144 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:26.148857 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 172/335] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:26.162470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c863bf-e1cf-4410-a743-f1bbb9e54f63 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:26.162949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c863bf-e1cf-4410-a743-f1bbb9e54f63 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:26.163025 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c863bf-e1cf-4410-a743-f1bbb9e54f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:26.190143 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-c0c863bf-e1cf-4410-a743-f1bbb9e54f63 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:26.191337 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c0c863bf-e1cf-4410-a743-f1bbb9e54f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:26.245710 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:26.275818 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 164/336] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:26.313563 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f4a23-a6ee-4cc9-ac60-e4d598bf1144 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:26.366538 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:26.454996 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f4a23-a6ee-4cc9-ac60-e4d598bf1144 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:26.499919 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e33ce-bfe8-4f5c-88d4-703286acf4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:26.500561 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e33ce-bfe8-4f5c-88d4-703286acf4f1 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:26.500831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e33ce-bfe8-4f5c-88d4-703286acf4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:26.501101 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e33ce-bfe8-4f5c-88d4-703286acf4f1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:26.517719 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e33ce-bfe8-4f5c-88d4-703286acf4f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:26.522976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-4c3e33ce-bfe8-4f5c-88d4-703286acf4f1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 08:23:26.522976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:26.522976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:26.522976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:26.522976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:26.522976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:26.522976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:26.522976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:26.522976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:26.522976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:26.522976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:26.522976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:26.522976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:26.522976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:26.522976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:26.522976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:26.523837 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:26.523837 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:26.523837 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:26.523837 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:26.523837 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:26.523837 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:26.523837 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:26.523837 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:26.523837 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:26.523837 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:26.523837 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:26.523837 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:26.523837 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:26.523837 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:26.523837 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:26.523837 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:26.524646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:26.524646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:26.524646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:26.524646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:26.524646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:26.524646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:26.524646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:26.524646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:26.524646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:26.524646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:26.524646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 08:23:26.524646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:26.524646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 173/337] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 08:23:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:26.530109 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c863bf-e1cf-4410-a743-f1bbb9e54f63 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:26.560392 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e0baa-03cc-4e26-9c4d-663a45292638 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:26.560764 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e0baa-03cc-4e26-9c4d-663a45292638 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:26.561041 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e0baa-03cc-4e26-9c4d-663a45292638 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:26.563241 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 174/338] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:26.567971 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f7d79-acd0-4fe1-8c8a-ab92ea46fc00 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:26.568253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f7d79-acd0-4fe1-8c8a-ab92ea46fc00 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:26.568522 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f7d79-acd0-4fe1-8c8a-ab92ea46fc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:26.594970 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-fd1f7d79-acd0-4fe1-8c8a-ab92ea46fc00 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:26.596158 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fd1f7d79-acd0-4fe1-8c8a-ab92ea46fc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:26.723420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 175/339] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:26.732861 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c863bf-e1cf-4410-a743-f1bbb9e54f63 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:26.756937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f7d79-acd0-4fe1-8c8a-ab92ea46fc00 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:26.797883 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916234c-8f5d-428f-bf53-bc755dfa70f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:26.798507 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916234c-8f5d-428f-bf53-bc755dfa70f5 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:26.798791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916234c-8f5d-428f-bf53-bc755dfa70f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:26.799067 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916234c-8f5d-428f-bf53-bc755dfa70f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:26.799752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8916234c-8f5d-428f-bf53-bc755dfa70f5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:26.800054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8916234c-8f5d-428f-bf53-bc755dfa70f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:26.801688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:26.801799 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8916234c-8f5d-428f-bf53-bc755dfa70f5 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:26.812552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 165/340] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:23:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:26.818594 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916234c-8f5d-428f-bf53-bc755dfa70f5 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:26.907934 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:26.949136 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f7d79-acd0-4fe1-8c8a-ab92ea46fc00 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:27.007924 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd3699-e241-4d24-8427-520a2bbe4746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:27.008479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd3699-e241-4d24-8427-520a2bbe4746 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:27.013311 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd3699-e241-4d24-8427-520a2bbe4746 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:27.013618 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd3699-e241-4d24-8427-520a2bbe4746 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:27.035008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916234c-8f5d-428f-bf53-bc755dfa70f5 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:27.037095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd3699-e241-4d24-8427-520a2bbe4746 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:27.042170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-d2bd3699-e241-4d24-8427-520a2bbe4746 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:23:27.042170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:27.042170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:27.042170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:27.042170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:27.042170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:27.042170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:27.042170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:27.042170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:27.042170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:27.042170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:27.042170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:27.042170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:27.042170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:27.042170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:27.042170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:27.043092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:27.043092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:27.043092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:27.043092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:27.043092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:27.043092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:27.043092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:27.043092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:27.043092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:27.043092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:27.043092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:27.043092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:27.043092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:27.043092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:27.043092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:27.043092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:27.043882 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:27.043882 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:27.043882 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:27.043882 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:27.043882 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:27.043882 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:27.043882 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:27.043882 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:27.043882 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:27.043882 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:27.043882 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:23:27.043882 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:27.049256 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 176/341] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:23:27.066672 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd3699-e241-4d24-8427-520a2bbe4746 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:27.095502 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b4314-eb83-429f-9601-0c8b7903a17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:27.096148 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b4314-eb83-429f-9601-0c8b7903a17b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:27.096416 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b4314-eb83-429f-9601-0c8b7903a17b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:27.096685 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b4314-eb83-429f-9601-0c8b7903a17b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:27.223144 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b4314-eb83-429f-9601-0c8b7903a17b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:27.224572 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b4314-eb83-429f-9601-0c8b7903a17b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:27.239743 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:27.272538 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd3699-e241-4d24-8427-520a2bbe4746 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:27.280321 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:27.335015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74154d7c-d6d6-4618-b128-7c530709582a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:27.336081 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74154d7c-d6d6-4618-b128-7c530709582a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:27.336562 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74154d7c-d6d6-4618-b128-7c530709582a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:27.337098 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74154d7c-d6d6-4618-b128-7c530709582a None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:27.338017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74154d7c-d6d6-4618-b128-7c530709582a None admin] RBAC: Adding query filter params (name=member) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:27.338568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74154d7c-d6d6-4618-b128-7c530709582a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:27.340335 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74154d7c-d6d6-4618-b128-7c530709582a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:27.350853 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 166/342] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:23:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:23:27.352085 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba0212-56ed-46b2-99b9-f9cfc63b7ce7 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:27.352364 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba0212-56ed-46b2-99b9-f9cfc63b7ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:27.352636 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba0212-56ed-46b2-99b9-f9cfc63b7ce7 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:27.354134 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 167/344] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:27.354239 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 177/344] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 08:23:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:27.360601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74154d7c-d6d6-4618-b128-7c530709582a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:27.360832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebe3f0-3acd-40d5-8b0b-99b4d7d6b902 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:27.361031 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebe3f0-3acd-40d5-8b0b-99b4d7d6b902 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:27.361351 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebe3f0-3acd-40d5-8b0b-99b4d7d6b902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:27.381832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-51ebe3f0-3acd-40d5-8b0b-99b4d7d6b902 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:27.383116 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-51ebe3f0-3acd-40d5-8b0b-99b4d7d6b902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:27.479497 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 168/345] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:27.480599 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebe3f0-3acd-40d5-8b0b-99b4d7d6b902 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:27.528756 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74154d7c-d6d6-4618-b128-7c530709582a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:27.570497 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:27.571506 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130a4fd-ece6-4a71-9811-d9fd7a5248ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:27.572122 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130a4fd-ece6-4a71-9811-d9fd7a5248ff None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:27.572394 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130a4fd-ece6-4a71-9811-d9fd7a5248ff None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:27.572668 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130a4fd-ece6-4a71-9811-d9fd7a5248ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:27.592495 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:27.610116 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebe3f0-3acd-40d5-8b0b-99b4d7d6b902 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:27.654155 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16640f9-9d44-4676-8a97-8cba46380908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:27.654802 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16640f9-9d44-4676-8a97-8cba46380908 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:27.655077 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16640f9-9d44-4676-8a97-8cba46380908 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:27.655348 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16640f9-9d44-4676-8a97-8cba46380908 None admin] PATH_INFO: `/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:27.656804 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16640f9-9d44-4676-8a97-8cba46380908 None admin] RBAC: Authorizing `identity:get_project(project_id=ce0954e79d28403bb7e0c4f0b78e4394)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:27.658277 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16640f9-9d44-4676-8a97-8cba46380908 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:27.660537 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 169/346] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:27 2022] GET /identity/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:27.661670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748f0bb-973c-498b-a379-f7b09bf66986 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:27.661958 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748f0bb-973c-498b-a379-f7b09bf66986 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:27.662213 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748f0bb-973c-498b-a379-f7b09bf66986 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:27.663406 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 170/347] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:27.664744 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d4cfa-b81f-43b0-99fe-8ce8a001bef5 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:27.664744 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d4cfa-b81f-43b0-99fe-8ce8a001bef5 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:27.665262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d4cfa-b81f-43b0-99fe-8ce8a001bef5 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:27.667327 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 171/348] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:27.668019 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d4cfa-b81f-43b0-99fe-8ce8a001bef5 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:27.684498 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0130a4fd-ece6-4a71-9811-d9fd7a5248ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:27.685989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0130a4fd-ece6-4a71-9811-d9fd7a5248ff None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:27.791755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d4cfa-b81f-43b0-99fe-8ce8a001bef5 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:27.815748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 178/349] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:23:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:23:27.816691 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130a4fd-ece6-4a71-9811-d9fd7a5248ff None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:27.837516 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ab008-9880-49aa-9b8d-0dc7bf9d2299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:27.838301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ab008-9880-49aa-9b8d-0dc7bf9d2299 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:27.838577 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ab008-9880-49aa-9b8d-0dc7bf9d2299 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:27.838979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ab008-9880-49aa-9b8d-0dc7bf9d2299 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:27.847428 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:27.854433 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ab008-9880-49aa-9b8d-0dc7bf9d2299 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:27.860079 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-de0ab008-9880-49aa-9b8d-0dc7bf9d2299 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:23:27.860079 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:27.860079 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:27.860079 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:27.860079 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:27.860079 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:27.860079 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:27.860079 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:27.860079 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:27.860079 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:27.860079 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:27.860079 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:27.860079 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:27.860079 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:27.860079 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:27.860079 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:27.862050 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:27.862050 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:27.862050 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:27.862050 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:27.862050 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:27.862050 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:27.862050 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:27.862050 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:27.862050 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:27.862050 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:27.862050 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:27.862050 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:27.862050 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:27.862050 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:27.862050 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:27.862050 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:27.862966 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:27.862966 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:27.862966 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:27.862966 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:27.862966 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:27.862966 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:27.862966 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:27.862966 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:27.862966 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:27.862966 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:27.862966 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:23:27.862966 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:27.862966 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 172/350] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:27.862966 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa868f-10f4-4d32-9edf-5a8d6b1c65e5 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:27.863688 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa868f-10f4-4d32-9edf-5a8d6b1c65e5 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:27.863688 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa868f-10f4-4d32-9edf-5a8d6b1c65e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:27.879445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:27.883249 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-71fa868f-10f4-4d32-9edf-5a8d6b1c65e5 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:27.884676 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-71fa868f-10f4-4d32-9edf-5a8d6b1c65e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:27.944493 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130a4fd-ece6-4a71-9811-d9fd7a5248ff None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:27.976235 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 173/351] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:27.978095 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9900c-14b1-4deb-9adf-a5dd80b113bd None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:27.978555 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9900c-14b1-4deb-9adf-a5dd80b113bd None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:27.979025 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9900c-14b1-4deb-9adf-a5dd80b113bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:27.986120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abcbd0e-8fea-4c8b-b06f-8a583720598c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:27.986884 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcbd0e-8fea-4c8b-b06f-8a583720598c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:27.987370 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcbd0e-8fea-4c8b-b06f-8a583720598c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:27.987735 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcbd0e-8fea-4c8b-b06f-8a583720598c None admin] PATH_INFO: `/v3/groups/c071340c0d454c1488b9dccd5561b053` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:27.994022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abcbd0e-8fea-4c8b-b06f-8a583720598c None admin] RBAC: Authorizing `identity:get_group(group_id=c071340c0d454c1488b9dccd5561b053)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:27.995621 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abcbd0e-8fea-4c8b-b06f-8a583720598c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:27.997655 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 179/352] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 08:23:27 2022] GET /identity/v3/groups/c071340c0d454c1488b9dccd5561b053 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:27.998434 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-cae9900c-14b1-4deb-9adf-a5dd80b113bd None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:27.998549 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abcbd0e-8fea-4c8b-b06f-8a583720598c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:28.000417 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-cae9900c-14b1-4deb-9adf-a5dd80b113bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:28.092065 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 174/353] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:28.092813 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9900c-14b1-4deb-9adf-a5dd80b113bd None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:28.128008 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abcbd0e-8fea-4c8b-b06f-8a583720598c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:28.138799 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:28.160892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:28.168505 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2fc92-344e-4e33-8578-60e343184a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:28.169373 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2fc92-344e-4e33-8578-60e343184a23 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:28.170659 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2fc92-344e-4e33-8578-60e343184a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:28.170933 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2fc92-344e-4e33-8578-60e343184a23 None admin] PATH_INFO: `/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:28.172614 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2fc92-344e-4e33-8578-60e343184a23 None admin] RBAC: Authorizing `identity:get_project(project_id=ce0954e79d28403bb7e0c4f0b78e4394)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:28.174101 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2fc92-344e-4e33-8578-60e343184a23 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:28.177437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 180/354] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:27 2022] GET /identity/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:28.178585 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2fc92-344e-4e33-8578-60e343184a23 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:28.216546 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9900c-14b1-4deb-9adf-a5dd80b113bd None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:28.258987 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4fa0fd-473e-4974-ac3e-9b8a4f37fa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:28.259641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4fa0fd-473e-4974-ac3e-9b8a4f37fa82 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:28.259866 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4fa0fd-473e-4974-ac3e-9b8a4f37fa82 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:28.260134 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4fa0fd-473e-4974-ac3e-9b8a4f37fa82 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:28.260810 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4fa0fd-473e-4974-ac3e-9b8a4f37fa82 None admin] RBAC: Adding query filter params (name=member) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:28.261110 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4fa0fd-473e-4974-ac3e-9b8a4f37fa82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:28.262543 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4fa0fd-473e-4974-ac3e-9b8a4f37fa82 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:28.273652 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 175/355] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 08:23:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:28.274637 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4fa0fd-473e-4974-ac3e-9b8a4f37fa82 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:28.299189 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2fc92-344e-4e33-8578-60e343184a23 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:28.337848 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3066784-005f-4288-9306-9e96c2531f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:28.338469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3066784-005f-4288-9306-9e96c2531f02 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:28.338754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3066784-005f-4288-9306-9e96c2531f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:28.339130 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3066784-005f-4288-9306-9e96c2531f02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:28.339967 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3066784-005f-4288-9306-9e96c2531f02 None admin] RBAC: Adding query filter params (group.id=c071340c0d454c1488b9dccd5561b053, role.id=152f4082bb67479ebaeb08c2fa683399, scope.project.id=ce0954e79d28403bb7e0c4f0b78e4394) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:28.340275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3066784-005f-4288-9306-9e96c2531f02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:28.341919 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3066784-005f-4288-9306-9e96c2531f02 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:28.349707 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 181/356] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Aug 8 08:23:28 2022] GET /identity/v3/role_assignments?group.id=c071340c0d454c1488b9dccd5561b053&scope.project.id=ce0954e79d28403bb7e0c4f0b78e4394&role.id=152f4082bb67479ebaeb08c2fa683399 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:28.350843 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5955c72-0eee-403f-849b-0df132f8ab75 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:28.351133 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5955c72-0eee-403f-849b-0df132f8ab75 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:28.351409 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5955c72-0eee-403f-849b-0df132f8ab75 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:28.352655 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 182/357] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:28.353574 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5955c72-0eee-403f-849b-0df132f8ab75 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:28.399120 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4fa0fd-473e-4974-ac3e-9b8a4f37fa82 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:28.409539 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:28.434568 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:28.459974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed8a37-4379-4458-be48-b1df66c0a783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:28.461110 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed8a37-4379-4458-be48-b1df66c0a783 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:28.461422 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed8a37-4379-4458-be48-b1df66c0a783 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:28.461697 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed8a37-4379-4458-be48-b1df66c0a783 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:28.481210 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed8a37-4379-4458-be48-b1df66c0a783 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:28.487082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-75ed8a37-4379-4458-be48-b1df66c0a783 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 08:23:28.487082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:28.487082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:28.487082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:28.487082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:28.487082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:28.487082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:28.487082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:28.487082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:28.487082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:28.487082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:28.487082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:28.487082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:28.487082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:28.487082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:28.487082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:28.488051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:28.488051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:28.488051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:28.488051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:28.488051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:28.488051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:28.488051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:28.488051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:28.488051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:28.488051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:28.488051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:28.488051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:28.488051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:28.488051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:28.488051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:28.488051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:28.488855 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:28.488855 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:28.488855 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:28.488855 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:28.488855 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:28.488855 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:28.488855 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:28.488855 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:28.488855 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:28.488855 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:28.488855 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 08:23:28.488855 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:28.488855 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 176/358] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 08:23:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:28.489559 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed8a37-4379-4458-be48-b1df66c0a783 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:28.501978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5955c72-0eee-403f-849b-0df132f8ab75 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:28.546591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d963f-1b70-4f77-8abd-e9197495f5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:28.547235 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d963f-1b70-4f77-8abd-e9197495f5a4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:28.547508 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d963f-1b70-4f77-8abd-e9197495f5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:28.547771 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d963f-1b70-4f77-8abd-e9197495f5a4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:28.561556 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d963f-1b70-4f77-8abd-e9197495f5a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:28.567496 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-9e5d963f-1b70-4f77-8abd-e9197495f5a4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 08:23:28.567496 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:28.567496 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:28.567496 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:28.567496 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:28.567496 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:28.567496 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:28.567496 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:28.567496 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:28.567496 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:28.567496 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:28.567496 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:28.567496 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:28.567496 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:28.567496 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:28.567496 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:28.568427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:28.568427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:28.568427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:28.568427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:28.568427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:28.568427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:28.568427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:28.568427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:28.568427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:28.568427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:28.568427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:28.568427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:28.568427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:28.568427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:28.568427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:28.568427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:28.569238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:28.569238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:28.569238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:28.569238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:28.569238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:28.569238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:28.569238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:28.569238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:28.569238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:28.569238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:28.569238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 08:23:28.569238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:28.569238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 183/359] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:28.569904 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d963f-1b70-4f77-8abd-e9197495f5a4 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:28.633958 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed8a37-4379-4458-be48-b1df66c0a783 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:28.680204 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ecb3f-67a5-4e40-8621-7e1d27f27ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:28.680846 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ecb3f-67a5-4e40-8621-7e1d27f27ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:28.681120 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ecb3f-67a5-4e40-8621-7e1d27f27ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:28.681424 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ecb3f-67a5-4e40-8621-7e1d27f27ce4 None admin] PATH_INFO: `/v3/groups/c071340c0d454c1488b9dccd5561b053` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:28.683245 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ecb3f-67a5-4e40-8621-7e1d27f27ce4 None admin] RBAC: Authorizing `identity:get_group(group_id=c071340c0d454c1488b9dccd5561b053)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:28.684678 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ecb3f-67a5-4e40-8621-7e1d27f27ce4 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:28.686287 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 177/360] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 08:23:28 2022] GET /identity/v3/groups/c071340c0d454c1488b9dccd5561b053 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:28.687972 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c0956-8a34-4831-b68f-f9a57cfa5072 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:28.688310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c0956-8a34-4831-b68f-f9a57cfa5072 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:28.688641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c0956-8a34-4831-b68f-f9a57cfa5072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:28.708595 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d963f-1b70-4f77-8abd-e9197495f5a4 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:28.712608 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:28.715799 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-6c1c0956-8a34-4831-b68f-f9a57cfa5072 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:28.717017 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6c1c0956-8a34-4831-b68f-f9a57cfa5072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:28.760707 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda270d-bc4f-4571-bf02-1c05037ee610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:28.761354 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda270d-bc4f-4571-bf02-1c05037ee610 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:28.762224 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda270d-bc4f-4571-bf02-1c05037ee610 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:28.762224 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda270d-bc4f-4571-bf02-1c05037ee610 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:28.788407 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:28.826477 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 178/361] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:28.829445 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c0956-8a34-4831-b68f-f9a57cfa5072 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:28.861654 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfda270d-bc4f-4571-bf02-1c05037ee610 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:28.863453 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfda270d-bc4f-4571-bf02-1c05037ee610 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:28.943231 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c0956-8a34-4831-b68f-f9a57cfa5072 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:28.984679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7452c-182f-4c00-af3b-bef5613c4804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:28.985462 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7452c-182f-4c00-af3b-bef5613c4804 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:28.985744 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7452c-182f-4c00-af3b-bef5613c4804 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:28.986019 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7452c-182f-4c00-af3b-bef5613c4804 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:28.986694 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b7452c-182f-4c00-af3b-bef5613c4804 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:28.986997 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b7452c-182f-4c00-af3b-bef5613c4804 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:28.988368 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b7452c-182f-4c00-af3b-bef5613c4804 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:28.996896 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 184/362] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:23:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:23:28.999233 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda270d-bc4f-4571-bf02-1c05037ee610 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:28.999405 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 179/363] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:23:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:29.000342 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7452c-182f-4c00-af3b-bef5613c4804 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:29.006272 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:29.058552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:29.134769 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7452c-182f-4c00-af3b-bef5613c4804 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:29.150820 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda270d-bc4f-4571-bf02-1c05037ee610 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:29.184560 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015059eb-748c-419c-a598-d6768610b396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:29.185557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015059eb-748c-419c-a598-d6768610b396 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:29.186015 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015059eb-748c-419c-a598-d6768610b396 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:29.186424 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015059eb-748c-419c-a598-d6768610b396 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:29.187486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015059eb-748c-419c-a598-d6768610b396 None admin] RBAC: Adding query filter params (group.id=c071340c0d454c1488b9dccd5561b053, role.id=b0ba6e2bf793402abe2e680c2a7dcb53, scope.project.id=ce0954e79d28403bb7e0c4f0b78e4394) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:29.188017 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015059eb-748c-419c-a598-d6768610b396 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:29.189916 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015059eb-748c-419c-a598-d6768610b396 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:29.191673 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954097d-1e07-4602-b927-a38fb2a08868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:29.192274 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954097d-1e07-4602-b927-a38fb2a08868 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:29.192556 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954097d-1e07-4602-b927-a38fb2a08868 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:29.192828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954097d-1e07-4602-b927-a38fb2a08868 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:29.193580 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954097d-1e07-4602-b927-a38fb2a08868 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:29.193847 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954097d-1e07-4602-b927-a38fb2a08868 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:29.195214 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954097d-1e07-4602-b927-a38fb2a08868 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:29.197630 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 180/364] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Aug 8 08:23:28 2022] GET /identity/v3/role_assignments?group.id=c071340c0d454c1488b9dccd5561b053&scope.project.id=ce0954e79d28403bb7e0c4f0b78e4394&role.id=b0ba6e2bf793402abe2e680c2a7dcb53 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:29.198678 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015059eb-748c-419c-a598-d6768610b396 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:29.216765 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 185/365] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 08:23:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:29.217499 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954097d-1e07-4602-b927-a38fb2a08868 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:29.319012 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:29.337056 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015059eb-748c-419c-a598-d6768610b396 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:29.338798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:29.363349 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954097d-1e07-4602-b927-a38fb2a08868 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:29.377580 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822b482-c2c6-4028-b595-060c738a5645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:29.378476 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822b482-c2c6-4028-b595-060c738a5645 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:29.378752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822b482-c2c6-4028-b595-060c738a5645 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:29.379093 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822b482-c2c6-4028-b595-060c738a5645 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:29.391788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5822b482-c2c6-4028-b595-060c738a5645 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:29.397708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-5822b482-c2c6-4028-b595-060c738a5645 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:23:29.397708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:29.397708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:29.397708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:29.397708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:29.397708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:29.397708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:29.397708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:29.397708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:29.397708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:29.397708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:29.397708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:29.397708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:29.397708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:29.397708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:29.397708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:29.398768 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:29.398768 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:29.398768 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:29.398768 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:29.398768 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:29.398768 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:29.398768 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:29.398768 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:29.398768 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:29.398768 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:29.398768 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:29.398768 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:29.398768 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:29.398768 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:29.398768 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:29.398768 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:29.399749 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:29.399749 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:29.399749 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:29.399749 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:29.399749 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:29.399749 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:29.399749 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:29.399749 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:29.399749 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:29.399749 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:29.399749 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:23:29.399749 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:29.401562 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 181/366] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:23:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:29.402428 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822b482-c2c6-4028-b595-060c738a5645 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:29.404959 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e4caa-2cf0-4e1d-bd5d-818ee4b2cb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:29.406595 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e4caa-2cf0-4e1d-bd5d-818ee4b2cb8a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:29.407087 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e4caa-2cf0-4e1d-bd5d-818ee4b2cb8a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:29.407487 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e4caa-2cf0-4e1d-bd5d-818ee4b2cb8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:29.483133 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e4caa-2cf0-4e1d-bd5d-818ee4b2cb8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:29.485007 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e4caa-2cf0-4e1d-bd5d-818ee4b2cb8a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:29.516386 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822b482-c2c6-4028-b595-060c738a5645 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:29.551680 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:29.557063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b814f-1eb5-4457-9c91-a40629dfdd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:29.558211 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b814f-1eb5-4457-9c91-a40629dfdd10 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:29.564794 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b814f-1eb5-4457-9c91-a40629dfdd10 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:29.564794 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b814f-1eb5-4457-9c91-a40629dfdd10 None admin] PATH_INFO: `/v3/projects/3ab93f396edd405c96c6effd6d5edb41` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:29.564794 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848b814f-1eb5-4457-9c91-a40629dfdd10 None admin] RBAC: Authorizing `identity:get_project(project_id=3ab93f396edd405c96c6effd6d5edb41)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:29.564794 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848b814f-1eb5-4457-9c91-a40629dfdd10 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:29.564794 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 182/367] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:29 2022] GET /identity/v3/projects/3ab93f396edd405c96c6effd6d5edb41 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:29.565707 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b814f-1eb5-4457-9c91-a40629dfdd10 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:29.596626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 186/368] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:23:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:23:29.598370 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e4caa-2cf0-4e1d-bd5d-818ee4b2cb8a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:29.606686 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:29.687640 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b814f-1eb5-4457-9c91-a40629dfdd10 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:29.717663 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e4caa-2cf0-4e1d-bd5d-818ee4b2cb8a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:29.735335 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f08832-87dd-4784-afe4-06cecfbbc2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:29.736312 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f08832-87dd-4784-afe4-06cecfbbc2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:29.736591 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f08832-87dd-4784-afe4-06cecfbbc2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:29.736862 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f08832-87dd-4784-afe4-06cecfbbc2e6 None admin] PATH_INFO: `/v3/users/f4ac3dc48fda4923b9e96bf8ce95537d` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:29.738061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f08832-87dd-4784-afe4-06cecfbbc2e6 None admin] RBAC: Authorizing `identity:get_user(user_id=f4ac3dc48fda4923b9e96bf8ce95537d)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:29.739677 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f08832-87dd-4784-afe4-06cecfbbc2e6 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:29.741706 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 183/369] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:23:29 2022] GET /identity/v3/users/f4ac3dc48fda4923b9e96bf8ce95537d => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:29.745746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f08832-87dd-4784-afe4-06cecfbbc2e6 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:29.773756 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eec08a-060b-4a44-a440-840b80b546b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:29.774716 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eec08a-060b-4a44-a440-840b80b546b8 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:29.775210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eec08a-060b-4a44-a440-840b80b546b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:29.775670 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eec08a-060b-4a44-a440-840b80b546b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:29.776615 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eec08a-060b-4a44-a440-840b80b546b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:29.777268 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eec08a-060b-4a44-a440-840b80b546b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:29.778873 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eec08a-060b-4a44-a440-840b80b546b8 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:29.789876 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 187/370] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:29.791220 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eec08a-060b-4a44-a440-840b80b546b8 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:29.825555 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:29.854350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:29.859108 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f08832-87dd-4784-afe4-06cecfbbc2e6 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:29.894918 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c97dc-88dd-469d-ba91-d46b869b8494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:29.895615 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c97dc-88dd-469d-ba91-d46b869b8494 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:29.895900 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c97dc-88dd-469d-ba91-d46b869b8494 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:29.896173 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c97dc-88dd-469d-ba91-d46b869b8494 None admin] PATH_INFO: `/v3/groups/c071340c0d454c1488b9dccd5561b053` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:29.897319 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169c97dc-88dd-469d-ba91-d46b869b8494 None admin] RBAC: Authorizing `identity:get_group(group_id=c071340c0d454c1488b9dccd5561b053)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:29.898746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169c97dc-88dd-469d-ba91-d46b869b8494 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:29.900383 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 184/371] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 08:23:29 2022] GET /identity/v3/groups/c071340c0d454c1488b9dccd5561b053 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:29.901168 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c97dc-88dd-469d-ba91-d46b869b8494 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:29.913535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eec08a-060b-4a44-a440-840b80b546b8 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:29.957633 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d31055-5e20-4f65-a55d-ed99e4257c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:29.958332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d31055-5e20-4f65-a55d-ed99e4257c8d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:29.958853 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d31055-5e20-4f65-a55d-ed99e4257c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:29.958853 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d31055-5e20-4f65-a55d-ed99e4257c8d None admin] PATH_INFO: `/v3/projects/3ab93f396edd405c96c6effd6d5edb41` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:29.960054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d31055-5e20-4f65-a55d-ed99e4257c8d None admin] RBAC: Authorizing `identity:get_project(project_id=3ab93f396edd405c96c6effd6d5edb41)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:29.961624 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d31055-5e20-4f65-a55d-ed99e4257c8d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:29.963219 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 188/372] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:29 2022] GET /identity/v3/projects/3ab93f396edd405c96c6effd6d5edb41 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:29.963977 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d31055-5e20-4f65-a55d-ed99e4257c8d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:30.031731 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c97dc-88dd-469d-ba91-d46b869b8494 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:30.074681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19b3dd-640f-4050-85fa-64c4566b3036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:30.075825 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19b3dd-640f-4050-85fa-64c4566b3036 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:30.076306 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19b3dd-640f-4050-85fa-64c4566b3036 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:30.076628 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:30.076935 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19b3dd-640f-4050-85fa-64c4566b3036 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:30.078877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e19b3dd-640f-4050-85fa-64c4566b3036 None admin] RBAC: Adding query filter params (role.id=ab682c3121c04a5fb600dc47750d013c, user.id=f4ac3dc48fda4923b9e96bf8ce95537d, scope.system=all) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:30.079728 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e19b3dd-640f-4050-85fa-64c4566b3036 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:30.082097 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e19b3dd-640f-4050-85fa-64c4566b3036 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:30.084641 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d31055-5e20-4f65-a55d-ed99e4257c8d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:30.098043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 185/373] 10.222.0.33 () {56 vars in 1308 bytes} [Mon Aug 8 08:23:29 2022] GET /identity/v3/role_assignments?user.id=f4ac3dc48fda4923b9e96bf8ce95537d&scope.system=all&role.id=ab682c3121c04a5fb600dc47750d013c => generated 510 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:30.099557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19b3dd-640f-4050-85fa-64c4566b3036 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:30.141268 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86cd76c-5d9b-48d7-8b37-7758d9e937cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:30.142388 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cd76c-5d9b-48d7-8b37-7758d9e937cc None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:30.143004 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cd76c-5d9b-48d7-8b37-7758d9e937cc None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:30.144328 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cd76c-5d9b-48d7-8b37-7758d9e937cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:30.167548 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:30.272655 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86cd76c-5d9b-48d7-8b37-7758d9e937cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:30.274629 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86cd76c-5d9b-48d7-8b37-7758d9e937cc None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:30.287096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19b3dd-640f-4050-85fa-64c4566b3036 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:30.324920 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78abc624-d7a3-4fcb-9ee6-048769d27c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:30.325562 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abc624-d7a3-4fcb-9ee6-048769d27c24 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:30.325844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abc624-d7a3-4fcb-9ee6-048769d27c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:30.326118 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abc624-d7a3-4fcb-9ee6-048769d27c24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:30.326847 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78abc624-d7a3-4fcb-9ee6-048769d27c24 None admin] RBAC: Adding query filter params (group.id=c071340c0d454c1488b9dccd5561b053, role.id=b0ba6e2bf793402abe2e680c2a7dcb53, scope.project.id=3ab93f396edd405c96c6effd6d5edb41) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:30.327155 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78abc624-d7a3-4fcb-9ee6-048769d27c24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:30.328758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78abc624-d7a3-4fcb-9ee6-048769d27c24 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:30.334627 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 186/374] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Aug 8 08:23:30 2022] GET /identity/v3/role_assignments?group.id=c071340c0d454c1488b9dccd5561b053&scope.project.id=3ab93f396edd405c96c6effd6d5edb41&role.id=b0ba6e2bf793402abe2e680c2a7dcb53 => generated 260 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:30.335974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78abc624-d7a3-4fcb-9ee6-048769d27c24 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:30.395553 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:30.406840 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 189/375] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:23:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:23:30.410019 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86cd76c-5d9b-48d7-8b37-7758d9e937cc None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:30.527723 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:30.548757 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78abc624-d7a3-4fcb-9ee6-048769d27c24 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:30.580618 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86cd76c-5d9b-48d7-8b37-7758d9e937cc None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:30.592438 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721aff72-2d2a-466e-8618-aa3ccb0bb147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:30.593976 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721aff72-2d2a-466e-8618-aa3ccb0bb147 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:30.594259 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721aff72-2d2a-466e-8618-aa3ccb0bb147 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:30.594537 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721aff72-2d2a-466e-8618-aa3ccb0bb147 None admin] PATH_INFO: `/v3/groups/c071340c0d454c1488b9dccd5561b053` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:30.596063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721aff72-2d2a-466e-8618-aa3ccb0bb147 None admin] RBAC: Authorizing `identity:get_group(group_id=c071340c0d454c1488b9dccd5561b053)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:30.598151 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721aff72-2d2a-466e-8618-aa3ccb0bb147 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:30.599961 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 187/376] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 08:23:30 2022] GET /identity/v3/groups/c071340c0d454c1488b9dccd5561b053 => generated 226 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:30.605698 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721aff72-2d2a-466e-8618-aa3ccb0bb147 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:30.623417 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16119a-e9c5-47ad-a982-d40d742f20a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:30.624065 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16119a-e9c5-47ad-a982-d40d742f20a8 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:30.624437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16119a-e9c5-47ad-a982-d40d742f20a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:30.624711 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16119a-e9c5-47ad-a982-d40d742f20a8 None admin] PATH_INFO: `/v3/projects/785791bda4044663bfe6cd25439a8358` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:30.625995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16119a-e9c5-47ad-a982-d40d742f20a8 None admin] RBAC: Authorizing `identity:get_project(project_id=785791bda4044663bfe6cd25439a8358)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:30.627563 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16119a-e9c5-47ad-a982-d40d742f20a8 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:30.629460 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 190/377] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:30 2022] GET /identity/v3/projects/785791bda4044663bfe6cd25439a8358 => generated 344 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:30.634824 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16119a-e9c5-47ad-a982-d40d742f20a8 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:30.719500 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:30.730033 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721aff72-2d2a-466e-8618-aa3ccb0bb147 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:30.775255 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16119a-e9c5-47ad-a982-d40d742f20a8 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:30.777952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0502553-21bc-4f18-b70f-92ffb7b69c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:30.778565 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0502553-21bc-4f18-b70f-92ffb7b69c74 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:30.778846 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0502553-21bc-4f18-b70f-92ffb7b69c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:30.779118 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0502553-21bc-4f18-b70f-92ffb7b69c74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:30.779853 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0502553-21bc-4f18-b70f-92ffb7b69c74 None admin] RBAC: Adding query filter params (group.id=c071340c0d454c1488b9dccd5561b053, role.id=152f4082bb67479ebaeb08c2fa683399, scope.project.id=3ab93f396edd405c96c6effd6d5edb41) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:30.780161 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0502553-21bc-4f18-b70f-92ffb7b69c74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:30.782250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0502553-21bc-4f18-b70f-92ffb7b69c74 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:30.789521 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 188/378] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Aug 8 08:23:30 2022] GET /identity/v3/role_assignments?group.id=c071340c0d454c1488b9dccd5561b053&scope.project.id=3ab93f396edd405c96c6effd6d5edb41&role.id=152f4082bb67479ebaeb08c2fa683399 => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:30.790761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54bfa02-3860-48bd-b414-5dfc63060a71 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:30.791051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54bfa02-3860-48bd-b414-5dfc63060a71 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:30.791327 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54bfa02-3860-48bd-b414-5dfc63060a71 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:30.792819 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 189/379] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:30.799423 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707fcc4-e55e-40c2-a2f6-ada753f3b3a3 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:30.799693 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707fcc4-e55e-40c2-a2f6-ada753f3b3a3 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:30.799891 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707fcc4-e55e-40c2-a2f6-ada753f3b3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:30.824036 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940823a1-61e5-4ba7-af9e-f5996e560704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:30.824700 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940823a1-61e5-4ba7-af9e-f5996e560704 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:30.824982 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940823a1-61e5-4ba7-af9e-f5996e560704 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:30.825288 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940823a1-61e5-4ba7-af9e-f5996e560704 None admin] PATH_INFO: `/v3/groups/9fa2c82d9d994f3f89e691f6b706993f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:30.831945 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940823a1-61e5-4ba7-af9e-f5996e560704 None admin] RBAC: Authorizing `identity:get_group(group_id=9fa2c82d9d994f3f89e691f6b706993f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:30.833575 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940823a1-61e5-4ba7-af9e-f5996e560704 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:30.836869 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 191/380] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 08:23:30 2022] GET /identity/v3/groups/9fa2c82d9d994f3f89e691f6b706993f => generated 229 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:30.844308 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940823a1-61e5-4ba7-af9e-f5996e560704 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:30.849171 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-0707fcc4-e55e-40c2-a2f6-ada753f3b3a3 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:30.850394 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0707fcc4-e55e-40c2-a2f6-ada753f3b3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:30.888112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:30.968865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940823a1-61e5-4ba7-af9e-f5996e560704 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:31.024012 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:31.037868 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926b354-6b78-4585-9e2e-f51f8756b6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:31.038467 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 190/381] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:23:31.038601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926b354-6b78-4585-9e2e-f51f8756b6b7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:31.038767 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926b354-6b78-4585-9e2e-f51f8756b6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:31.039050 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926b354-6b78-4585-9e2e-f51f8756b6b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:31.039886 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926b354-6b78-4585-9e2e-f51f8756b6b7 None admin] RBAC: Adding query filter params (group.id=9fa2c82d9d994f3f89e691f6b706993f, role.id=c1e514a4b5424f959b069f36cbfd8147, scope.project.id=785791bda4044663bfe6cd25439a8358) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:31.040201 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926b354-6b78-4585-9e2e-f51f8756b6b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:31.045759 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926b354-6b78-4585-9e2e-f51f8756b6b7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:31.053202 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 192/382] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Aug 8 08:23:30 2022] GET /identity/v3/role_assignments?group.id=9fa2c82d9d994f3f89e691f6b706993f&scope.project.id=785791bda4044663bfe6cd25439a8358&role.id=c1e514a4b5424f959b069f36cbfd8147 => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:31.063837 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707fcc4-e55e-40c2-a2f6-ada753f3b3a3 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:31.187734 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707fcc4-e55e-40c2-a2f6-ada753f3b3a3 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:31.228555 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9876be-1d45-452b-8f55-d298f55c5798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:31.229237 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9876be-1d45-452b-8f55-d298f55c5798 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:31.229516 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9876be-1d45-452b-8f55-d298f55c5798 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:31.229781 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9876be-1d45-452b-8f55-d298f55c5798 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:31.243759 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9876be-1d45-452b-8f55-d298f55c5798 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:31.248748 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-be9876be-1d45-452b-8f55-d298f55c5798 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 08:23:31.248748 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:31.248748 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:31.248748 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:31.248748 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:31.248748 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:31.248748 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:31.248748 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:31.248748 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:31.248748 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:31.248748 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:31.248748 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:31.248748 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:31.248748 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:31.248748 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:31.248748 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:31.249769 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:31.249769 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:31.249769 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:31.249769 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:31.249769 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:31.249769 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:31.249769 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:31.249769 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:31.249769 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:31.249769 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:31.249769 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:31.249769 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:31.249769 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:31.249769 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:31.249769 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:31.249769 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:31.250573 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:31.250573 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:31.250573 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:31.250573 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:31.250573 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:31.250573 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:31.250573 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:31.250573 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:31.250573 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:31.250573 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:31.250573 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 08:23:31.250573 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:31.255141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 191/383] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 08:23:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:23:31.257617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926b354-6b78-4585-9e2e-f51f8756b6b7 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:31.368826 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926b354-6b78-4585-9e2e-f51f8756b6b7 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:31.412545 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f4d6b-d353-48b5-9c50-dcd9a42ddb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:31.413184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f4d6b-d353-48b5-9c50-dcd9a42ddb3f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:31.413436 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f4d6b-d353-48b5-9c50-dcd9a42ddb3f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:31.415286 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f4d6b-d353-48b5-9c50-dcd9a42ddb3f None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:31.415286 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f4d6b-d353-48b5-9c50-dcd9a42ddb3f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:31.415286 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f4d6b-d353-48b5-9c50-dcd9a42ddb3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:31.419244 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f4d6b-d353-48b5-9c50-dcd9a42ddb3f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:31.432384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 193/384] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:23:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:31.433761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9876be-1d45-452b-8f55-d298f55c5798 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:31.458884 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:31.597692 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95654124-aaf3-453b-9388-da281bfa5ca5 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:31.598058 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95654124-aaf3-453b-9388-da281bfa5ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:31.598299 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95654124-aaf3-453b-9388-da281bfa5ca5 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:31.599763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 194/385] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 08:23:31.605174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4beb2-bf09-4667-8bc9-9e5c58bb96b4 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:31.605514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4beb2-bf09-4667-8bc9-9e5c58bb96b4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:31.605796 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4beb2-bf09-4667-8bc9-9e5c58bb96b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:31.634966 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-2bc4beb2-bf09-4667-8bc9-9e5c58bb96b4 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:31.636088 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2bc4beb2-bf09-4667-8bc9-9e5c58bb96b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:31.667127 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9876be-1d45-452b-8f55-d298f55c5798 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:31.719930 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:31.747591 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22901c2c-9f11-4980-a36f-15adc2d11eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:31.748466 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22901c2c-9f11-4980-a36f-15adc2d11eaa None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:31.748910 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22901c2c-9f11-4980-a36f-15adc2d11eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:31.749887 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22901c2c-9f11-4980-a36f-15adc2d11eaa None admin] PATH_INFO: `/v3/groups/c071340c0d454c1488b9dccd5561b053` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:31.752592 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22901c2c-9f11-4980-a36f-15adc2d11eaa None admin] RBAC: Authorizing `identity:get_group(group_id=c071340c0d454c1488b9dccd5561b053)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:31.757965 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22901c2c-9f11-4980-a36f-15adc2d11eaa None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:31.766766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 192/386] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 08:23:31 2022] GET /identity/v3/groups/c071340c0d454c1488b9dccd5561b053 => generated 226 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:31.767198 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 195/387] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:23:31.770332 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22901c2c-9f11-4980-a36f-15adc2d11eaa None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:31.815703 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc4beb2-bf09-4667-8bc9-9e5c58bb96b4 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:31.942814 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:31.970085 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc4beb2-bf09-4667-8bc9-9e5c58bb96b4 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:32.006939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2564dc-54b5-40a3-bf60-2223843c005e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:32.007964 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2564dc-54b5-40a3-bf60-2223843c005e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:32.008447 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2564dc-54b5-40a3-bf60-2223843c005e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:32.008908 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2564dc-54b5-40a3-bf60-2223843c005e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:32.020404 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2564dc-54b5-40a3-bf60-2223843c005e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:32.025514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-0c2564dc-54b5-40a3-bf60-2223843c005e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:23:32.025514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:32.025514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:32.025514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:32.025514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:32.025514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:32.025514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:32.025514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:32.025514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:32.025514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:32.025514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:32.025514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:32.025514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:32.025514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:32.025514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:32.025514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:32.026364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:32.026364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:32.026364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:32.026364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:32.026364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:32.026364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:32.026364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:32.026364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:32.026364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:32.026364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:32.026364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:32.026364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:32.026364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:32.026364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:32.026364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:32.026364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:32.027186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:32.027186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:32.027186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:32.027186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:32.027186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:32.027186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:32.027186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:32.027186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:32.027186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:32.027186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:32.027186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:23:32.027186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:32.034473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 196/388] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:23:32.035886 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2564dc-54b5-40a3-bf60-2223843c005e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:32.060601 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22901c2c-9f11-4980-a36f-15adc2d11eaa None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:32.086014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:32.146635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2564dc-54b5-40a3-bf60-2223843c005e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:32.192692 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191cfc4-1b97-4f0e-ac89-a70c406aab4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:32.193887 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191cfc4-1b97-4f0e-ac89-a70c406aab4e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:32.194443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191cfc4-1b97-4f0e-ac89-a70c406aab4e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:32.194904 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191cfc4-1b97-4f0e-ac89-a70c406aab4e None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:32.195770 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7191cfc4-1b97-4f0e-ac89-a70c406aab4e None admin] RBAC: Adding query filter params (name=member) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:32.196257 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7191cfc4-1b97-4f0e-ac89-a70c406aab4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:32.197841 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237aa23-bd89-400c-a032-f6d99a2181d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:32.198141 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7191cfc4-1b97-4f0e-ac89-a70c406aab4e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:32.198494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237aa23-bd89-400c-a032-f6d99a2181d8 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:32.198773 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237aa23-bd89-400c-a032-f6d99a2181d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:32.199044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237aa23-bd89-400c-a032-f6d99a2181d8 None admin] PATH_INFO: `/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:32.200181 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237aa23-bd89-400c-a032-f6d99a2181d8 None admin] RBAC: Authorizing `identity:get_project(project_id=ce0954e79d28403bb7e0c4f0b78e4394)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:32.201763 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237aa23-bd89-400c-a032-f6d99a2181d8 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:32.208703 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 193/389] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:31 2022] GET /identity/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394 => generated 298 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:32.215059 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 197/390] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 08:23:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:32.220521 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237aa23-bd89-400c-a032-f6d99a2181d8 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:32.220946 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191cfc4-1b97-4f0e-ac89-a70c406aab4e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:32.347055 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191cfc4-1b97-4f0e-ac89-a70c406aab4e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:32.349948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:32.364858 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237aa23-bd89-400c-a032-f6d99a2181d8 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:32.397041 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35cbcc-70f6-4bd7-aa3d-4584614cb289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:32.398218 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35cbcc-70f6-4bd7-aa3d-4584614cb289 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:32.398833 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35cbcc-70f6-4bd7-aa3d-4584614cb289 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:32.399348 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35cbcc-70f6-4bd7-aa3d-4584614cb289 None admin] PATH_INFO: `/v3/groups/c071340c0d454c1488b9dccd5561b053` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:32.400004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ae7494-6b50-4cc8-afd4-ea03bbfc2dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:32.400729 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae7494-6b50-4cc8-afd4-ea03bbfc2dee None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:32.401074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae7494-6b50-4cc8-afd4-ea03bbfc2dee None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:32.401317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35cbcc-70f6-4bd7-aa3d-4584614cb289 None admin] RBAC: Authorizing `identity:get_group(group_id=c071340c0d454c1488b9dccd5561b053)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:32.401459 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae7494-6b50-4cc8-afd4-ea03bbfc2dee None admin] PATH_INFO: `/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394/groups/c071340c0d454c1488b9dccd5561b053/roles/152f4082bb67479ebaeb08c2fa683399` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:32.403349 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35cbcc-70f6-4bd7-aa3d-4584614cb289 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:32.404991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ae7494-6b50-4cc8-afd4-ea03bbfc2dee None admin] RBAC: Authorizing `identity:create_grant(project_id=ce0954e79d28403bb7e0c4f0b78e4394, group_id=c071340c0d454c1488b9dccd5561b053, role_id=152f4082bb67479ebaeb08c2fa683399)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:32.410331 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ae7494-6b50-4cc8-afd4-ea03bbfc2dee None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:32.410886 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 198/391] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 08:23:32 2022] GET /identity/v3/groups/c071340c0d454c1488b9dccd5561b053 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:32.415605 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35cbcc-70f6-4bd7-aa3d-4584614cb289 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:32.480384 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 194/392] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Aug 8 08:23:32 2022] PUT /identity/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394/groups/c071340c0d454c1488b9dccd5561b053/roles/152f4082bb67479ebaeb08c2fa683399 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:23:32.542284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35cbcc-70f6-4bd7-aa3d-4584614cb289 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:32.580766 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a1531-77c4-40c8-861e-db7e19daaebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:32.581689 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a1531-77c4-40c8-861e-db7e19daaebc None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:32.581969 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a1531-77c4-40c8-861e-db7e19daaebc None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:32.582244 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a1531-77c4-40c8-861e-db7e19daaebc None admin] PATH_INFO: `/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:32.583412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a1531-77c4-40c8-861e-db7e19daaebc None admin] RBAC: Authorizing `identity:get_project(project_id=ce0954e79d28403bb7e0c4f0b78e4394)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:32.585012 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a1531-77c4-40c8-861e-db7e19daaebc None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:32.587009 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 199/393] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:32 2022] GET /identity/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:32.593490 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ae7494-6b50-4cc8-afd4-ea03bbfc2dee None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:32.608959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:32.745251 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ae7494-6b50-4cc8-afd4-ea03bbfc2dee None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:32.795166 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44654f7-f5cc-45bc-be19-6471b19fd432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:32.795848 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44654f7-f5cc-45bc-be19-6471b19fd432 None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:32.796655 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44654f7-f5cc-45bc-be19-6471b19fd432 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:32.796971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44654f7-f5cc-45bc-be19-6471b19fd432 None admin] PATH_INFO: `/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394/groups/c071340c0d454c1488b9dccd5561b053/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:32.798927 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44654f7-f5cc-45bc-be19-6471b19fd432 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce0954e79d28403bb7e0c4f0b78e4394, group_id=c071340c0d454c1488b9dccd5561b053, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:32.800371 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44654f7-f5cc-45bc-be19-6471b19fd432 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:32.875292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 195/394] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Aug 8 08:23:32 2022] PUT /identity/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394/groups/c071340c0d454c1488b9dccd5561b053/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:23:32.953345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20392a5f-3d23-4fae-aca7-7952f2093a74 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:32.953669 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20392a5f-3d23-4fae-aca7-7952f2093a74 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:32.953900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20392a5f-3d23-4fae-aca7-7952f2093a74 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:32.955511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 200/395] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:32.961494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7940305-6768-4cb4-8701-09530670b801 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:32.961790 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7940305-6768-4cb4-8701-09530670b801 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:32.962119 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7940305-6768-4cb4-8701-09530670b801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:32.993538 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-b7940305-6768-4cb4-8701-09530670b801 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:32.994843 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b7940305-6768-4cb4-8701-09530670b801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:33.080197 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:33.090274 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 196/396] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:33.113957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20392a5f-3d23-4fae-aca7-7952f2093a74 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:33.166196 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:33.249015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20392a5f-3d23-4fae-aca7-7952f2093a74 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:33.297747 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ad3b2-81ea-40ef-b454-a9fd5b8c774a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:33.298727 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ad3b2-81ea-40ef-b454-a9fd5b8c774a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:33.299303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ad3b2-81ea-40ef-b454-a9fd5b8c774a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:33.299762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ad3b2-81ea-40ef-b454-a9fd5b8c774a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:33.315038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ad3b2-81ea-40ef-b454-a9fd5b8c774a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:33.320577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-c08ad3b2-81ea-40ef-b454-a9fd5b8c774a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:23:33.320577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:33.320577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:33.320577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:33.320577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:33.320577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:33.320577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:33.320577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:33.320577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:33.320577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:33.320577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:33.320577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:33.320577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:33.320577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:33.320577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:33.320577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:33.321501 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:33.321501 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:33.321501 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:33.321501 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:33.321501 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:33.321501 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:33.321501 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:33.321501 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:33.321501 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:33.321501 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:33.321501 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:33.321501 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:33.321501 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:33.321501 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:33.321501 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:33.321501 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:33.322276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:33.322276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:33.322276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:33.322276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:33.322276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:33.322276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:33.322276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:33.322276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:33.322276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:33.322276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:33.322276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:23:33.322276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:33.324912 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 201/397] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:33.330462 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7940305-6768-4cb4-8701-09530670b801 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:33.496868 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7940305-6768-4cb4-8701-09530670b801 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:33.538899 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574884c3-e588-4080-b57a-3d68ad935138 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:33.539432 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574884c3-e588-4080-b57a-3d68ad935138 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:33.539649 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574884c3-e588-4080-b57a-3d68ad935138 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:33.541184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 202/398] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:33.546900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137ebfc-2ff3-4bdc-aa3c-0191284eccce None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:33.547187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137ebfc-2ff3-4bdc-aa3c-0191284eccce None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:33.547461 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137ebfc-2ff3-4bdc-aa3c-0191284eccce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:33.547578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331883d-8f1f-44ee-92fb-90fc68eb461a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:33.548118 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331883d-8f1f-44ee-92fb-90fc68eb461a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:33.548397 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331883d-8f1f-44ee-92fb-90fc68eb461a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:33.548667 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331883d-8f1f-44ee-92fb-90fc68eb461a None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:33.549368 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7331883d-8f1f-44ee-92fb-90fc68eb461a None admin] RBAC: Adding query filter params (name=member) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:33.549671 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7331883d-8f1f-44ee-92fb-90fc68eb461a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:33.551039 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7331883d-8f1f-44ee-92fb-90fc68eb461a None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:33.563401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 197/399] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 08:23:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:33.569019 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331883d-8f1f-44ee-92fb-90fc68eb461a None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:33.579777 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8137ebfc-2ff3-4bdc-aa3c-0191284eccce None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:33.584383 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8137ebfc-2ff3-4bdc-aa3c-0191284eccce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:33.659931 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:33.701311 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 203/400] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:23:33.703650 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3219f-1d7a-461c-b0e3-e77dd7b56ea1 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:33.704167 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3219f-1d7a-461c-b0e3-e77dd7b56ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:33.704674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3219f-1d7a-461c-b0e3-e77dd7b56ea1 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:33.710711 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 204/401] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:33.714662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331883d-8f1f-44ee-92fb-90fc68eb461a None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:33.720125 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51258bda-bf79-400c-b1f3-669137e33c87 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:33.720410 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51258bda-bf79-400c-b1f3-669137e33c87 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:33.720697 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51258bda-bf79-400c-b1f3-669137e33c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:33.742354 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-51258bda-bf79-400c-b1f3-669137e33c87 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:33.744072 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-51258bda-bf79-400c-b1f3-669137e33c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:33.757829 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79717f33-726f-4c12-91eb-f880ab616c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:33.758577 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79717f33-726f-4c12-91eb-f880ab616c45 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:33.758861 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79717f33-726f-4c12-91eb-f880ab616c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:33.759134 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79717f33-726f-4c12-91eb-f880ab616c45 None admin] PATH_INFO: `/v3/groups/c071340c0d454c1488b9dccd5561b053` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:33.760325 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79717f33-726f-4c12-91eb-f880ab616c45 None admin] RBAC: Authorizing `identity:get_group(group_id=c071340c0d454c1488b9dccd5561b053)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:33.761933 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79717f33-726f-4c12-91eb-f880ab616c45 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:33.763912 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 198/402] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 08:23:33 2022] GET /identity/v3/groups/c071340c0d454c1488b9dccd5561b053 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:33.764709 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79717f33-726f-4c12-91eb-f880ab616c45 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:33.774143 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:33.833713 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 205/403] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:23:33.835499 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51258bda-bf79-400c-b1f3-669137e33c87 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:33.884534 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79717f33-726f-4c12-91eb-f880ab616c45 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:33.919334 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:33.931384 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478dca60-67af-4e3e-8c04-398a631fa384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:33.932026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478dca60-67af-4e3e-8c04-398a631fa384 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:33.932308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478dca60-67af-4e3e-8c04-398a631fa384 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:33.932580 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478dca60-67af-4e3e-8c04-398a631fa384 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:33.947965 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478dca60-67af-4e3e-8c04-398a631fa384 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:33.955027 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-478dca60-67af-4e3e-8c04-398a631fa384 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:23:33.955027 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:33.955027 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:33.955027 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:33.955027 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:33.955027 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:33.955027 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:33.955027 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:33.955027 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:33.955027 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:33.955027 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:33.955027 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:33.955027 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:33.955027 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:33.955027 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:33.955027 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:33.956402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:33.956402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:33.956402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:33.956402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:33.956402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:33.956402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:33.956402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:33.956402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:33.956402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:33.956402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:33.956402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:33.956402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:33.956402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:33.956402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:33.956402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:33.956402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:33.957275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:33.957275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:33.957275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:33.957275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:33.957275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:33.957275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:33.957275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:33.957275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:33.957275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:33.957275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:33.957275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:23:33.957275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:33.957275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 199/404] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:23:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:33.957936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478dca60-67af-4e3e-8c04-398a631fa384 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:33.958652 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51258bda-bf79-400c-b1f3-669137e33c87 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:34.000494 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a9ef1-e99f-41ed-b2fa-7ddca6479e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:34.001858 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a9ef1-e99f-41ed-b2fa-7ddca6479e4b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:34.002450 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a9ef1-e99f-41ed-b2fa-7ddca6479e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:34.002942 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a9ef1-e99f-41ed-b2fa-7ddca6479e4b None admin] PATH_INFO: `/v3/projects/3ab93f396edd405c96c6effd6d5edb41` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:34.004488 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a9ef1-e99f-41ed-b2fa-7ddca6479e4b None admin] RBAC: Authorizing `identity:get_project(project_id=3ab93f396edd405c96c6effd6d5edb41)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:34.006270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a9ef1-e99f-41ed-b2fa-7ddca6479e4b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:34.008327 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 206/405] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:33 2022] GET /identity/v3/projects/3ab93f396edd405c96c6effd6d5edb41 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:34.009492 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a9ef1-e99f-41ed-b2fa-7ddca6479e4b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:34.048294 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:34.080026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478dca60-67af-4e3e-8c04-398a631fa384 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:34.117972 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71b780-75b8-4a67-b8f3-4f42290127ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:34.119172 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71b780-75b8-4a67-b8f3-4f42290127ce None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:34.119559 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71b780-75b8-4a67-b8f3-4f42290127ce None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:34.119937 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71b780-75b8-4a67-b8f3-4f42290127ce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:34.133342 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b71b780-75b8-4a67-b8f3-4f42290127ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:34.138860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a9ef1-e99f-41ed-b2fa-7ddca6479e4b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:34.140516 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-0b71b780-75b8-4a67-b8f3-4f42290127ce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 08:23:34.140516 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:34.140516 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:34.140516 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:34.140516 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:34.140516 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:34.140516 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:34.140516 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:34.140516 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:34.140516 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:34.140516 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:34.140516 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:34.140516 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:34.140516 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:34.140516 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:34.140516 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:34.141509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:34.141509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:34.141509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:34.141509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:34.141509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:34.141509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:34.141509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:34.141509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:34.141509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:34.141509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:34.141509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:34.141509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:34.141509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:34.141509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:34.141509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:34.141509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:34.142302 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:34.142302 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:34.142302 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:34.142302 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:34.142302 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:34.142302 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:34.142302 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:34.142302 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:34.142302 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:34.142302 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:34.142302 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 08:23:34.142302 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:34.145208 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 200/406] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 08:23:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:34.146188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71b780-75b8-4a67-b8f3-4f42290127ce None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:34.186243 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a89be-cf17-47e4-bbcf-09d3ea4758e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:34.187058 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a89be-cf17-47e4-bbcf-09d3ea4758e9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:34.187366 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a89be-cf17-47e4-bbcf-09d3ea4758e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:34.187657 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a89be-cf17-47e4-bbcf-09d3ea4758e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:34.188322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a89be-cf17-47e4-bbcf-09d3ea4758e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:34.188628 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a89be-cf17-47e4-bbcf-09d3ea4758e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:34.190027 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a89be-cf17-47e4-bbcf-09d3ea4758e9 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:34.199024 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 207/407] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:34.199829 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a89be-cf17-47e4-bbcf-09d3ea4758e9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:34.205495 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:34.263342 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71b780-75b8-4a67-b8f3-4f42290127ce None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:34.300419 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301f0d2-6b23-4c35-9f29-530c0b846da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:34.301528 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301f0d2-6b23-4c35-9f29-530c0b846da3 None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:34.301909 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301f0d2-6b23-4c35-9f29-530c0b846da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:34.302289 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301f0d2-6b23-4c35-9f29-530c0b846da3 None admin] PATH_INFO: `/v3/projects/3ab93f396edd405c96c6effd6d5edb41/groups/c071340c0d454c1488b9dccd5561b053/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:34.303628 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:34.305172 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6301f0d2-6b23-4c35-9f29-530c0b846da3 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ab93f396edd405c96c6effd6d5edb41, group_id=c071340c0d454c1488b9dccd5561b053, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:34.307164 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6301f0d2-6b23-4c35-9f29-530c0b846da3 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:34.313893 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a89be-cf17-47e4-bbcf-09d3ea4758e9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:34.351807 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e11dc09-801e-4dbc-ba99-5cd21484699f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:34.352350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11dc09-801e-4dbc-ba99-5cd21484699f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:34.352630 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11dc09-801e-4dbc-ba99-5cd21484699f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:34.352934 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11dc09-801e-4dbc-ba99-5cd21484699f None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:34.353657 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e11dc09-801e-4dbc-ba99-5cd21484699f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:34.353959 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e11dc09-801e-4dbc-ba99-5cd21484699f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:34.355387 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e11dc09-801e-4dbc-ba99-5cd21484699f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:34.369681 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 208/408] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:23:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:34.369681 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e11dc09-801e-4dbc-ba99-5cd21484699f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:34.381652 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 201/409] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Aug 8 08:23:34 2022] PUT /identity/v3/projects/3ab93f396edd405c96c6effd6d5edb41/groups/c071340c0d454c1488b9dccd5561b053/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:23:34.382356 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301f0d2-6b23-4c35-9f29-530c0b846da3 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:34.517067 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e11dc09-801e-4dbc-ba99-5cd21484699f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:34.559763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c02a12-7a23-457b-bf37-2cf7bcb51760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:34.561293 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c02a12-7a23-457b-bf37-2cf7bcb51760 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:34.562317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c02a12-7a23-457b-bf37-2cf7bcb51760 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:34.562923 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c02a12-7a23-457b-bf37-2cf7bcb51760 None admin] PATH_INFO: `/v3/groups/9fa2c82d9d994f3f89e691f6b706993f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:34.564493 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c02a12-7a23-457b-bf37-2cf7bcb51760 None admin] RBAC: Authorizing `identity:get_group(group_id=9fa2c82d9d994f3f89e691f6b706993f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:34.566210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c02a12-7a23-457b-bf37-2cf7bcb51760 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:34.569010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 209/410] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 08:23:34 2022] GET /identity/v3/groups/9fa2c82d9d994f3f89e691f6b706993f => generated 229 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:34.574397 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c02a12-7a23-457b-bf37-2cf7bcb51760 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:34.602012 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:34.643806 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301f0d2-6b23-4c35-9f29-530c0b846da3 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:34.654065 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:34.681499 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47b494-9e01-4717-a414-0272dc059a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:34.682396 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47b494-9e01-4717-a414-0272dc059a4f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:34.682396 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47b494-9e01-4717-a414-0272dc059a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:34.682671 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47b494-9e01-4717-a414-0272dc059a4f None admin] PATH_INFO: `/v3/groups/c071340c0d454c1488b9dccd5561b053` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:34.683837 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47b494-9e01-4717-a414-0272dc059a4f None admin] RBAC: Authorizing `identity:get_group(group_id=c071340c0d454c1488b9dccd5561b053)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:34.685423 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47b494-9e01-4717-a414-0272dc059a4f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:34.688245 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 202/411] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 08:23:34 2022] GET /identity/v3/groups/c071340c0d454c1488b9dccd5561b053 => generated 226 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:34.694624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47b494-9e01-4717-a414-0272dc059a4f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:34.717360 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c02a12-7a23-457b-bf37-2cf7bcb51760 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:34.758617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3665d0-f592-420a-98e4-4ee3c4901170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:34.759666 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3665d0-f592-420a-98e4-4ee3c4901170 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:34.760225 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3665d0-f592-420a-98e4-4ee3c4901170 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:34.760647 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3665d0-f592-420a-98e4-4ee3c4901170 None admin] PATH_INFO: `/v3/projects/785791bda4044663bfe6cd25439a8358` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:34.762135 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3665d0-f592-420a-98e4-4ee3c4901170 None admin] RBAC: Authorizing `identity:get_project(project_id=785791bda4044663bfe6cd25439a8358)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:34.764209 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3665d0-f592-420a-98e4-4ee3c4901170 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:34.766537 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 210/412] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:34 2022] GET /identity/v3/projects/785791bda4044663bfe6cd25439a8358 => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:34.773186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3665d0-f592-420a-98e4-4ee3c4901170 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:34.870746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47b494-9e01-4717-a414-0272dc059a4f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:34.934099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f4d3a0-504a-4474-b439-b9b59d9f31c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:34.935086 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4d3a0-504a-4474-b439-b9b59d9f31c2 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:34.935444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4d3a0-504a-4474-b439-b9b59d9f31c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:34.935794 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4d3a0-504a-4474-b439-b9b59d9f31c2 None admin] PATH_INFO: `/v3/projects/3ab93f396edd405c96c6effd6d5edb41` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:34.937041 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f4d3a0-504a-4474-b439-b9b59d9f31c2 None admin] RBAC: Authorizing `identity:get_project(project_id=3ab93f396edd405c96c6effd6d5edb41)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:34.938659 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f4d3a0-504a-4474-b439-b9b59d9f31c2 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:34.940788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 203/413] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:34 2022] GET /identity/v3/projects/3ab93f396edd405c96c6effd6d5edb41 => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:34.942179 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557639c2-aa9d-48fa-bbd4-35d38068fabf None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:34.942562 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557639c2-aa9d-48fa-bbd4-35d38068fabf None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:34.942934 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557639c2-aa9d-48fa-bbd4-35d38068fabf None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:34.944489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 204/414] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:34.945504 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:34.947155 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557639c2-aa9d-48fa-bbd4-35d38068fabf None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:34.950296 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3665d0-f592-420a-98e4-4ee3c4901170 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:34.999873 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f320c663-7406-4443-8abe-e2770d751359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:35.000673 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320c663-7406-4443-8abe-e2770d751359 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:35.001600 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320c663-7406-4443-8abe-e2770d751359 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:35.001998 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320c663-7406-4443-8abe-e2770d751359 None admin] PATH_INFO: `/v3/projects/785791bda4044663bfe6cd25439a8358/groups/9fa2c82d9d994f3f89e691f6b706993f/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:35.005127 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:35.006333 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f320c663-7406-4443-8abe-e2770d751359 None admin] RBAC: Authorizing `identity:create_grant(project_id=785791bda4044663bfe6cd25439a8358, group_id=9fa2c82d9d994f3f89e691f6b706993f, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:35.008251 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f320c663-7406-4443-8abe-e2770d751359 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:35.067916 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557639c2-aa9d-48fa-bbd4-35d38068fabf None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:35.100173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 211/415] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Aug 8 08:23:34 2022] PUT /identity/v3/projects/785791bda4044663bfe6cd25439a8358/groups/9fa2c82d9d994f3f89e691f6b706993f/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:23:35.101211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bac7c3-dd5b-4d67-8ba4-344894b5d2de None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:35.101508 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bac7c3-dd5b-4d67-8ba4-344894b5d2de None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:35.102799 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bac7c3-dd5b-4d67-8ba4-344894b5d2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:35.123623 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39d8fb-9fd0-4fbf-bfa7-759135aab793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:35.125251 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39d8fb-9fd0-4fbf-bfa7-759135aab793 None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:35.127596 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39d8fb-9fd0-4fbf-bfa7-759135aab793 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:35.129121 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39d8fb-9fd0-4fbf-bfa7-759135aab793 None admin] PATH_INFO: `/v3/projects/3ab93f396edd405c96c6effd6d5edb41/groups/c071340c0d454c1488b9dccd5561b053/roles/152f4082bb67479ebaeb08c2fa683399` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:35.136860 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b39d8fb-9fd0-4fbf-bfa7-759135aab793 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ab93f396edd405c96c6effd6d5edb41, group_id=c071340c0d454c1488b9dccd5561b053, role_id=152f4082bb67479ebaeb08c2fa683399)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:35.141433 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b39d8fb-9fd0-4fbf-bfa7-759135aab793 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:35.148957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-32bac7c3-dd5b-4d67-8ba4-344894b5d2de None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:35.150186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-32bac7c3-dd5b-4d67-8ba4-344894b5d2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:35.226617 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 205/416] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Aug 8 08:23:34 2022] PUT /identity/v3/projects/3ab93f396edd405c96c6effd6d5edb41/groups/c071340c0d454c1488b9dccd5561b053/roles/152f4082bb67479ebaeb08c2fa683399 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:23:35.232512 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 212/417] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:35.277102 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39d8fb-9fd0-4fbf-bfa7-759135aab793 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:35.332832 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b02fee-a266-4c3c-8322-364b98ccef85 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:35.333566 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b02fee-a266-4c3c-8322-364b98ccef85 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:35.334050 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b02fee-a266-4c3c-8322-364b98ccef85 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:35.336779 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 213/418] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:35.341180 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771d210-228f-4b24-909d-d038185cf96b None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:35.341752 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771d210-228f-4b24-909d-d038185cf96b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:35.342212 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771d210-228f-4b24-909d-d038185cf96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:35.364642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4771d210-228f-4b24-909d-d038185cf96b None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:35.366342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4771d210-228f-4b24-909d-d038185cf96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:35.393828 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39d8fb-9fd0-4fbf-bfa7-759135aab793 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:35.395546 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:35.445835 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7001e9-81f6-4008-b3ab-82bd662c0b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:35.446463 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7001e9-81f6-4008-b3ab-82bd662c0b8c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:35.446761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7001e9-81f6-4008-b3ab-82bd662c0b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:35.447001 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7001e9-81f6-4008-b3ab-82bd662c0b8c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:35.460727 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7001e9-81f6-4008-b3ab-82bd662c0b8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:35.462631 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 214/419] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:23:35.466766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-2c7001e9-81f6-4008-b3ab-82bd662c0b8c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 08:23:35.466766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:35.466766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:35.466766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:35.466766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:35.466766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:35.466766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:35.466766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:35.466766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:35.466766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:35.466766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:35.466766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:35.466766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:35.466766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:35.466766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:35.466766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:35.467660 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:35.467660 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:35.467660 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:35.467660 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:35.467660 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:35.467660 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:35.467660 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:35.467660 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:35.467660 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:35.467660 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:35.467660 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:35.467660 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:35.467660 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:35.467660 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:35.467660 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:35.467660 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:35.469129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:35.469129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:35.469129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:35.469129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:35.469129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:35.469129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:35.469129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:35.469129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:35.469129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:35.469129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:35.469129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 08:23:35.469129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:35.469129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 206/420] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 08:23:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:35.474412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771d210-228f-4b24-909d-d038185cf96b None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:35.485558 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:35.500037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7001e9-81f6-4008-b3ab-82bd662c0b8c None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:35.634389 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771d210-228f-4b24-909d-d038185cf96b None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:35.676642 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7001e9-81f6-4008-b3ab-82bd662c0b8c None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:35.701941 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb1e60-eb4f-4534-a530-87e653672fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:35.703265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb1e60-eb4f-4534-a530-87e653672fce None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:35.703685 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb1e60-eb4f-4534-a530-87e653672fce None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:35.703959 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb1e60-eb4f-4534-a530-87e653672fce None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:35.704761 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb1e60-eb4f-4534-a530-87e653672fce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:35.705062 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb1e60-eb4f-4534-a530-87e653672fce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:35.706736 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb1e60-eb4f-4534-a530-87e653672fce None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:35.720996 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 215/421] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:23:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:35.722614 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33777424-06b5-466b-8e4f-70f06cfc13a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:35.723400 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33777424-06b5-466b-8e4f-70f06cfc13a8 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:35.723680 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33777424-06b5-466b-8e4f-70f06cfc13a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:35.723950 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33777424-06b5-466b-8e4f-70f06cfc13a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:35.726347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb1e60-eb4f-4534-a530-87e653672fce None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:35.753775 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33777424-06b5-466b-8e4f-70f06cfc13a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:35.762567 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-33777424-06b5-466b-8e4f-70f06cfc13a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:23:35.762567 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:35.762567 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:35.762567 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:35.762567 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:35.762567 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:35.762567 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:35.762567 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:35.762567 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:35.762567 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:35.762567 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:35.762567 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:35.762567 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:35.762567 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:35.762567 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:35.762567 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:35.763523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:35.763523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:35.763523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:35.763523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:35.763523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:35.763523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:35.763523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:35.763523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:35.763523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:35.763523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:35.763523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:35.763523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:35.763523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:35.763523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:35.763523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:35.763523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:35.764328 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:35.764328 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:35.764328 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:35.764328 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:35.764328 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:35.764328 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:35.764328 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:35.764328 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:35.764328 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:35.764328 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:35.764328 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:23:35.764328 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:35.767572 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 207/422] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:23:35.773467 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33777424-06b5-466b-8e4f-70f06cfc13a8 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:35.816342 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:35.819739 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:35.852502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb1e60-eb4f-4534-a530-87e653672fce None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:35.886876 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9d90d-20a2-4ba5-824c-4d6f6cc208b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:35.887804 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9d90d-20a2-4ba5-824c-4d6f6cc208b5 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:35.888226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9d90d-20a2-4ba5-824c-4d6f6cc208b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:35.888666 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9d90d-20a2-4ba5-824c-4d6f6cc208b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:35.915350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33777424-06b5-466b-8e4f-70f06cfc13a8 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:35.961270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a9d90d-20a2-4ba5-824c-4d6f6cc208b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:35.963187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a9d90d-20a2-4ba5-824c-4d6f6cc208b5 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:35.989876 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15edfa66-71ae-4288-b4b4-28d5688cd1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:35.990462 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15edfa66-71ae-4288-b4b4-28d5688cd1f8 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:35.990747 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15edfa66-71ae-4288-b4b4-28d5688cd1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:35.991024 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15edfa66-71ae-4288-b4b4-28d5688cd1f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:35.991705 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15edfa66-71ae-4288-b4b4-28d5688cd1f8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:35.992026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15edfa66-71ae-4288-b4b4-28d5688cd1f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:35.993419 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15edfa66-71ae-4288-b4b4-28d5688cd1f8 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:36.010610 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 208/423] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 08:23:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:36.016065 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15edfa66-71ae-4288-b4b4-28d5688cd1f8 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:36.036757 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:36.099058 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 216/424] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:23:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 08:23:36.105774 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9d90d-20a2-4ba5-824c-4d6f6cc208b5 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:36.199981 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:36.208503 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9d90d-20a2-4ba5-824c-4d6f6cc208b5 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:36.228972 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15edfa66-71ae-4288-b4b4-28d5688cd1f8 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:36.248628 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5887e-91a3-4ede-bb5b-64a80a4b66b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:36.249387 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5887e-91a3-4ede-bb5b-64a80a4b66b0 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:36.249625 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5887e-91a3-4ede-bb5b-64a80a4b66b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:36.249893 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5887e-91a3-4ede-bb5b-64a80a4b66b0 None admin] PATH_INFO: `/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:36.251095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e5887e-91a3-4ede-bb5b-64a80a4b66b0 None admin] RBAC: Authorizing `identity:get_project(project_id=ce0954e79d28403bb7e0c4f0b78e4394)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:36.252550 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e5887e-91a3-4ede-bb5b-64a80a4b66b0 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:36.255411 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 217/425] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:36 2022] GET /identity/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:36.259556 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5887e-91a3-4ede-bb5b-64a80a4b66b0 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:36.294947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803ee59-c6d6-4def-931e-a4ac4c35f932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:36.295471 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803ee59-c6d6-4def-931e-a4ac4c35f932 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:36.295749 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803ee59-c6d6-4def-931e-a4ac4c35f932 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:36.296174 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803ee59-c6d6-4def-931e-a4ac4c35f932 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:36.341480 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:36.404647 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803ee59-c6d6-4def-931e-a4ac4c35f932 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:36.406310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803ee59-c6d6-4def-931e-a4ac4c35f932 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:36.498625 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:36.519056 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 209/426] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:23:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 08:23:36.521800 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5887e-91a3-4ede-bb5b-64a80a4b66b0 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:36.524437 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803ee59-c6d6-4def-931e-a4ac4c35f932 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:36.569532 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a10fd-faf1-4e49-aa1c-b899f78b1daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:36.571371 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a10fd-faf1-4e49-aa1c-b899f78b1daf None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:36.571655 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a10fd-faf1-4e49-aa1c-b899f78b1daf None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:36.571920 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a10fd-faf1-4e49-aa1c-b899f78b1daf None admin] PATH_INFO: `/v3/groups/c071340c0d454c1488b9dccd5561b053` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:36.574271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2a10fd-faf1-4e49-aa1c-b899f78b1daf None admin] RBAC: Authorizing `identity:get_group(group_id=c071340c0d454c1488b9dccd5561b053)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:36.575730 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2a10fd-faf1-4e49-aa1c-b899f78b1daf None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:36.582965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 218/427] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 08:23:36 2022] GET /identity/v3/groups/c071340c0d454c1488b9dccd5561b053 => generated 226 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:36.590129 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a10fd-faf1-4e49-aa1c-b899f78b1daf None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:36.668774 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803ee59-c6d6-4def-931e-a4ac4c35f932 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:36.714248 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a10fd-faf1-4e49-aa1c-b899f78b1daf None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:36.721990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:36.735446 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef88785-54c1-4dda-85f1-28d456398f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:36.736088 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef88785-54c1-4dda-85f1-28d456398f70 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:36.736369 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef88785-54c1-4dda-85f1-28d456398f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:36.736641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef88785-54c1-4dda-85f1-28d456398f70 None admin] PATH_INFO: `/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:36.738957 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef88785-54c1-4dda-85f1-28d456398f70 None admin] RBAC: Authorizing `identity:get_project(project_id=ce0954e79d28403bb7e0c4f0b78e4394)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:36.739829 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef88785-54c1-4dda-85f1-28d456398f70 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:36.741869 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 210/428] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:36 2022] GET /identity/v3/projects/ce0954e79d28403bb7e0c4f0b78e4394 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:36.747324 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef88785-54c1-4dda-85f1-28d456398f70 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:36.775218 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1bc91-2e0d-4ed0-a926-3449b0f0a77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:36.776907 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1bc91-2e0d-4ed0-a926-3449b0f0a77c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:36.777666 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1bc91-2e0d-4ed0-a926-3449b0f0a77c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:36.778140 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1bc91-2e0d-4ed0-a926-3449b0f0a77c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:36.779808 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac1bc91-2e0d-4ed0-a926-3449b0f0a77c None admin] RBAC: Adding query filter params (group.id=c071340c0d454c1488b9dccd5561b053, role.id=152f4082bb67479ebaeb08c2fa683399, scope.project.id=ce0954e79d28403bb7e0c4f0b78e4394) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:36.781138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac1bc91-2e0d-4ed0-a926-3449b0f0a77c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:36.782880 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac1bc91-2e0d-4ed0-a926-3449b0f0a77c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:36.791145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 219/429] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Aug 8 08:23:36 2022] GET /identity/v3/role_assignments?group.id=c071340c0d454c1488b9dccd5561b053&scope.project.id=ce0954e79d28403bb7e0c4f0b78e4394&role.id=152f4082bb67479ebaeb08c2fa683399 => generated 611 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:36.793538 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b13fb-bcc9-4763-9349-32af8e004545 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:36.794035 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b13fb-bcc9-4763-9349-32af8e004545 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:36.794505 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b13fb-bcc9-4763-9349-32af8e004545 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:36.796105 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 220/430] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:36.808267 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d90db-64f9-45a4-9988-b92717750726 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:36.808570 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d90db-64f9-45a4-9988-b92717750726 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:36.808812 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d90db-64f9-45a4-9988-b92717750726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:36.814915 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:36.854373 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-902d90db-64f9-45a4-9988-b92717750726 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:36.855482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-902d90db-64f9-45a4-9988-b92717750726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:36.869528 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef88785-54c1-4dda-85f1-28d456398f70 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:36.909163 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13140751-f90f-4f56-a2e0-3af5846479f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:36.910325 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13140751-f90f-4f56-a2e0-3af5846479f6 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:36.910937 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13140751-f90f-4f56-a2e0-3af5846479f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:36.911439 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13140751-f90f-4f56-a2e0-3af5846479f6 None admin] PATH_INFO: `/v3/groups/c071340c0d454c1488b9dccd5561b053` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:36.912990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13140751-f90f-4f56-a2e0-3af5846479f6 None admin] RBAC: Authorizing `identity:get_group(group_id=c071340c0d454c1488b9dccd5561b053)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:36.914740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13140751-f90f-4f56-a2e0-3af5846479f6 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:36.916874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 211/431] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 08:23:36 2022] GET /identity/v3/groups/c071340c0d454c1488b9dccd5561b053 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:36.928895 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13140751-f90f-4f56-a2e0-3af5846479f6 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:37.004425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 221/432] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:37.043306 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d90db-64f9-45a4-9988-b92717750726 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:37.085784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13140751-f90f-4f56-a2e0-3af5846479f6 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:37.125521 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:37.142098 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690639b-dac8-4662-8f8d-6316fc97037d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:37.144904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690639b-dac8-4662-8f8d-6316fc97037d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:37.144904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690639b-dac8-4662-8f8d-6316fc97037d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:37.144904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690639b-dac8-4662-8f8d-6316fc97037d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:37.144904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5690639b-dac8-4662-8f8d-6316fc97037d None admin] RBAC: Adding query filter params (group.id=c071340c0d454c1488b9dccd5561b053, role.id=b0ba6e2bf793402abe2e680c2a7dcb53, scope.project.id=ce0954e79d28403bb7e0c4f0b78e4394) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:37.144904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5690639b-dac8-4662-8f8d-6316fc97037d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:37.145754 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5690639b-dac8-4662-8f8d-6316fc97037d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:37.148712 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:37.151889 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d90db-64f9-45a4-9988-b92717750726 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:37.152703 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 212/433] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Aug 8 08:23:36 2022] GET /identity/v3/role_assignments?group.id=c071340c0d454c1488b9dccd5561b053&scope.project.id=ce0954e79d28403bb7e0c4f0b78e4394&role.id=b0ba6e2bf793402abe2e680c2a7dcb53 => generated 611 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:37.192613 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c7d85-b5d9-42e6-a063-ce45f46c6f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:37.193665 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c7d85-b5d9-42e6-a063-ce45f46c6f17 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:37.194126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c7d85-b5d9-42e6-a063-ce45f46c6f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:37.194573 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c7d85-b5d9-42e6-a063-ce45f46c6f17 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:37.208253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c7d85-b5d9-42e6-a063-ce45f46c6f17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:37.213637 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-3d2c7d85-b5d9-42e6-a063-ce45f46c6f17 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:23:37.213637 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:37.213637 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:37.213637 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:37.213637 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:37.213637 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:37.213637 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:37.213637 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:37.213637 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:37.213637 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:37.213637 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:37.213637 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:37.213637 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:37.213637 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:37.213637 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:37.213637 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:37.214629 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:37.214629 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:37.214629 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:37.214629 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:37.214629 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:37.214629 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:37.214629 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:37.214629 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:37.214629 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:37.214629 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:37.214629 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:37.214629 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:37.214629 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:37.214629 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:37.214629 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:37.214629 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:37.215527 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:37.215527 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:37.215527 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:37.215527 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:37.215527 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:37.215527 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:37.215527 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:37.215527 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:37.215527 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:37.215527 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:37.215527 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 08:23:37.215527 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:37.218205 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 222/434] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:23:37.225376 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690639b-dac8-4662-8f8d-6316fc97037d None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:37.338183 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690639b-dac8-4662-8f8d-6316fc97037d None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:37.374401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27737ede-cd7f-4939-8316-e0992b6adf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:37.375070 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27737ede-cd7f-4939-8316-e0992b6adf48 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:37.375389 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27737ede-cd7f-4939-8316-e0992b6adf48 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:37.375718 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27737ede-cd7f-4939-8316-e0992b6adf48 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:37.376354 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27737ede-cd7f-4939-8316-e0992b6adf48 None admin] RBAC: Adding query filter params (name=member) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:37.376657 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27737ede-cd7f-4939-8316-e0992b6adf48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:37.378085 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27737ede-cd7f-4939-8316-e0992b6adf48 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:37.387321 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 213/435] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 08:23:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:37.394149 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c7d85-b5d9-42e6-a063-ce45f46c6f17 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:37.515028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c7d85-b5d9-42e6-a063-ce45f46c6f17 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:37.551086 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83c545-c36b-4e4d-85e7-1cfd20b39cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:37.551706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83c545-c36b-4e4d-85e7-1cfd20b39cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:37.551987 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83c545-c36b-4e4d-85e7-1cfd20b39cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:37.552259 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83c545-c36b-4e4d-85e7-1cfd20b39cb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:37.582599 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:37.598451 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740be89-f37f-4071-9e49-05f69a55f9e3 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:37.608703 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740be89-f37f-4071-9e49-05f69a55f9e3 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:37.608703 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740be89-f37f-4071-9e49-05f69a55f9e3 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:37.608703 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 214/436] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:37.608703 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29ff9f-3bd7-40a1-b4c2-bd38099737f0 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:37.608703 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29ff9f-3bd7-40a1-b4c2-bd38099737f0 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:37.608703 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29ff9f-3bd7-40a1-b4c2-bd38099737f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:37.628780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-6d29ff9f-3bd7-40a1-b4c2-bd38099737f0 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:37.629349 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83c545-c36b-4e4d-85e7-1cfd20b39cb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:37.630492 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6d29ff9f-3bd7-40a1-b4c2-bd38099737f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:37.630821 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83c545-c36b-4e4d-85e7-1cfd20b39cb5 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:37.695155 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:37.721059 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 215/437] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:37.722608 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f40d8d-6730-4087-9c28-be8f102b1a18 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:37.723063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f40d8d-6730-4087-9c28-be8f102b1a18 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:37.723503 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f40d8d-6730-4087-9c28-be8f102b1a18 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:37.724903 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 216/438] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:37.730555 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39df98-3cec-4d6c-af04-c155a09034b1 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:37.731095 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39df98-3cec-4d6c-af04-c155a09034b1 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:37.731646 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39df98-3cec-4d6c-af04-c155a09034b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:37.755250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-1c39df98-3cec-4d6c-af04-c155a09034b1 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:37.756832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1c39df98-3cec-4d6c-af04-c155a09034b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:37.766649 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 223/439] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:23:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:23:37.767273 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83c545-c36b-4e4d-85e7-1cfd20b39cb5 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:37.826432 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 217/440] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:23:37.827846 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39df98-3cec-4d6c-af04-c155a09034b1 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:37.832969 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:37.872531 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83c545-c36b-4e4d-85e7-1cfd20b39cb5 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:37.909751 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a79713-9531-4a98-be66-7aef1697aa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:37.909751 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a79713-9531-4a98-be66-7aef1697aa42 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:37.909751 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a79713-9531-4a98-be66-7aef1697aa42 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:37.909751 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a79713-9531-4a98-be66-7aef1697aa42 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:37.917864 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a79713-9531-4a98-be66-7aef1697aa42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:37.923071 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-e8a79713-9531-4a98-be66-7aef1697aa42 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 08:23:37.923071 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:37.923071 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:37.923071 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:37.923071 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:37.923071 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:37.923071 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:37.923071 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:37.923071 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:37.923071 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:37.923071 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:37.923071 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:37.923071 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:37.923071 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:37.923071 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:37.923071 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:37.924016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:37.924016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:37.924016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:37.924016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:37.924016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:37.924016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:37.924016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:37.924016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:37.924016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:37.924016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:37.924016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:37.924016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:37.924016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:37.924016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:37.924016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:37.924016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:37.924797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:37.924797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:37.924797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:37.924797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:37.924797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:37.924797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:37.924797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:37.924797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:37.924797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:37.924797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:37.924797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 08:23:37.924797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:37.924797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 224/441] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 08:23:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:37.926028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a79713-9531-4a98-be66-7aef1697aa42 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:37.926094 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39df98-3cec-4d6c-af04-c155a09034b1 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:37.935211 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:37.962281 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786a729-3a82-4ce2-a936-066a1cbec071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:37.962952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786a729-3a82-4ce2-a936-066a1cbec071 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:37.963228 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786a729-3a82-4ce2-a936-066a1cbec071 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:37.963502 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786a729-3a82-4ce2-a936-066a1cbec071 None admin] PATH_INFO: `/v3/projects/3ab93f396edd405c96c6effd6d5edb41` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:37.964605 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6786a729-3a82-4ce2-a936-066a1cbec071 None admin] RBAC: Authorizing `identity:get_project(project_id=3ab93f396edd405c96c6effd6d5edb41)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:37.966049 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6786a729-3a82-4ce2-a936-066a1cbec071 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:37.967648 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 218/442] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:37 2022] GET /identity/v3/projects/3ab93f396edd405c96c6effd6d5edb41 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:37.968469 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786a729-3a82-4ce2-a936-066a1cbec071 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:38.025062 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a79713-9531-4a98-be66-7aef1697aa42 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:38.044571 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:38.058631 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67b9ba-686b-4d68-83c6-892452145844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:38.059258 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67b9ba-686b-4d68-83c6-892452145844 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:38.059544 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67b9ba-686b-4d68-83c6-892452145844 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:38.059817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67b9ba-686b-4d68-83c6-892452145844 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:38.066909 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786a729-3a82-4ce2-a936-066a1cbec071 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:38.072007 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed67b9ba-686b-4d68-83c6-892452145844 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:38.077254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-ed67b9ba-686b-4d68-83c6-892452145844 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:23:38.077254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:38.077254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:38.077254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:38.077254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:38.077254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:38.077254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:38.077254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:38.077254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:38.077254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:38.077254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:38.077254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:38.077254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:38.077254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:38.077254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:38.077254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:38.078329 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:38.078329 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:38.078329 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:38.078329 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:38.078329 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:38.078329 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:38.078329 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:38.078329 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:38.078329 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:38.078329 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:38.078329 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:38.078329 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:38.078329 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:38.078329 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:38.078329 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:38.078329 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:38.079254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:38.079254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:38.079254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:38.079254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:38.079254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:38.079254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:38.079254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:38.079254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:38.079254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:38.079254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:38.079254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:23:38.079254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:38.079254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 225/443] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:23:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:38.080012 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67b9ba-686b-4d68-83c6-892452145844 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:38.106993 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600098b-5188-49d3-aa2b-46c8d0250ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:38.107590 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600098b-5188-49d3-aa2b-46c8d0250ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:38.107873 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600098b-5188-49d3-aa2b-46c8d0250ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:38.108137 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600098b-5188-49d3-aa2b-46c8d0250ec0 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:38.109466 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0600098b-5188-49d3-aa2b-46c8d0250ec0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:38.110023 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0600098b-5188-49d3-aa2b-46c8d0250ec0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:38.111272 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0600098b-5188-49d3-aa2b-46c8d0250ec0 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:38.118970 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 219/444] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:23:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:38.119737 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600098b-5188-49d3-aa2b-46c8d0250ec0 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:38.154569 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:38.183095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67b9ba-686b-4d68-83c6-892452145844 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:38.216848 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68e38c-4e2a-44b7-97e0-a59be00a5965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:38.217500 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68e38c-4e2a-44b7-97e0-a59be00a5965 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:38.217962 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68e38c-4e2a-44b7-97e0-a59be00a5965 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:38.218396 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68e38c-4e2a-44b7-97e0-a59be00a5965 None admin] PATH_INFO: `/v3/groups/c071340c0d454c1488b9dccd5561b053` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:38.219528 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68e38c-4e2a-44b7-97e0-a59be00a5965 None admin] RBAC: Authorizing `identity:get_group(group_id=c071340c0d454c1488b9dccd5561b053)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:38.220372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600098b-5188-49d3-aa2b-46c8d0250ec0 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:38.220956 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68e38c-4e2a-44b7-97e0-a59be00a5965 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:38.222697 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 226/445] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 08:23:38 2022] GET /identity/v3/groups/c071340c0d454c1488b9dccd5561b053 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:38.223364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68e38c-4e2a-44b7-97e0-a59be00a5965 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:38.254691 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178755d-6e8c-4533-a236-af6b96bd09c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:38.255295 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178755d-6e8c-4533-a236-af6b96bd09c9 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:38.255575 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178755d-6e8c-4533-a236-af6b96bd09c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:38.255847 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178755d-6e8c-4533-a236-af6b96bd09c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:38.256512 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d178755d-6e8c-4533-a236-af6b96bd09c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:38.256812 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d178755d-6e8c-4533-a236-af6b96bd09c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:38.258210 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d178755d-6e8c-4533-a236-af6b96bd09c9 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:38.271810 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:38.278577 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 220/446] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:38.278577 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178755d-6e8c-4533-a236-af6b96bd09c9 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:38.324107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68e38c-4e2a-44b7-97e0-a59be00a5965 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:38.357913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0783e2-8746-45c6-bf76-9e1b4abfc55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:38.358623 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0783e2-8746-45c6-bf76-9e1b4abfc55e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:38.358936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0783e2-8746-45c6-bf76-9e1b4abfc55e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:38.359209 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0783e2-8746-45c6-bf76-9e1b4abfc55e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:38.367541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:38.372487 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178755d-6e8c-4533-a236-af6b96bd09c9 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:38.407335 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f56e76-6e62-4f4c-9084-5c0f1fe1f6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:38.407945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f56e76-6e62-4f4c-9084-5c0f1fe1f6cb None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:38.408232 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f56e76-6e62-4f4c-9084-5c0f1fe1f6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:38.408508 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f56e76-6e62-4f4c-9084-5c0f1fe1f6cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:38.409276 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f56e76-6e62-4f4c-9084-5c0f1fe1f6cb None admin] RBAC: Adding query filter params (group.id=c071340c0d454c1488b9dccd5561b053, role.id=b0ba6e2bf793402abe2e680c2a7dcb53, scope.project.id=3ab93f396edd405c96c6effd6d5edb41) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:38.409588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f56e76-6e62-4f4c-9084-5c0f1fe1f6cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:38.411022 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f56e76-6e62-4f4c-9084-5c0f1fe1f6cb None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:38.416750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 221/447] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Aug 8 08:23:38 2022] GET /identity/v3/role_assignments?group.id=c071340c0d454c1488b9dccd5561b053&scope.project.id=3ab93f396edd405c96c6effd6d5edb41&role.id=b0ba6e2bf793402abe2e680c2a7dcb53 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:38.417671 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f56e76-6e62-4f4c-9084-5c0f1fe1f6cb None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:38.426621 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0783e2-8746-45c6-bf76-9e1b4abfc55e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:38.428124 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0783e2-8746-45c6-bf76-9e1b4abfc55e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:38.488828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:38.541167 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f56e76-6e62-4f4c-9084-5c0f1fe1f6cb None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:38.548582 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 227/448] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:23:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:23:38.554335 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0783e2-8746-45c6-bf76-9e1b4abfc55e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:38.585962 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d763d-47b0-4954-91ce-5c38448410f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:38.586580 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d763d-47b0-4954-91ce-5c38448410f8 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:38.586862 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d763d-47b0-4954-91ce-5c38448410f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:38.587130 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d763d-47b0-4954-91ce-5c38448410f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:38.624325 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:38.672213 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d763d-47b0-4954-91ce-5c38448410f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:38.674353 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d763d-47b0-4954-91ce-5c38448410f8 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:38.675192 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0783e2-8746-45c6-bf76-9e1b4abfc55e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:38.715769 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e7c3c-ee14-4197-857b-a043ca80bee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:38.716407 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e7c3c-ee14-4197-857b-a043ca80bee4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:38.716683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e7c3c-ee14-4197-857b-a043ca80bee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:38.716954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e7c3c-ee14-4197-857b-a043ca80bee4 None admin] PATH_INFO: `/v3/projects/3ab93f396edd405c96c6effd6d5edb41` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:38.718401 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09e7c3c-ee14-4197-857b-a043ca80bee4 None admin] RBAC: Authorizing `identity:get_project(project_id=3ab93f396edd405c96c6effd6d5edb41)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:38.720001 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09e7c3c-ee14-4197-857b-a043ca80bee4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:38.721875 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 228/449] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:38 2022] GET /identity/v3/projects/3ab93f396edd405c96c6effd6d5edb41 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:38.727746 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e7c3c-ee14-4197-857b-a043ca80bee4 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:38.747728 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:38.803939 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 222/450] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:23:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:23:38.809607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d763d-47b0-4954-91ce-5c38448410f8 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:38.832054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e7c3c-ee14-4197-857b-a043ca80bee4 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:38.865482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5136aa-7d32-45fb-97de-136d63b28cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:38.866108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5136aa-7d32-45fb-97de-136d63b28cab None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:38.866389 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5136aa-7d32-45fb-97de-136d63b28cab None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:38.866661 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5136aa-7d32-45fb-97de-136d63b28cab None admin] PATH_INFO: `/v3/groups/c071340c0d454c1488b9dccd5561b053` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:38.867774 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5136aa-7d32-45fb-97de-136d63b28cab None admin] RBAC: Authorizing `identity:get_group(group_id=c071340c0d454c1488b9dccd5561b053)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:38.869112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:38.869219 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5136aa-7d32-45fb-97de-136d63b28cab None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:38.870821 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 229/451] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 08:23:38 2022] GET /identity/v3/groups/c071340c0d454c1488b9dccd5561b053 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:38.876636 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5136aa-7d32-45fb-97de-136d63b28cab None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:38.911282 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d763d-47b0-4954-91ce-5c38448410f8 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:38.944655 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c016d9c-bb61-45dd-8690-f705e953b0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:38.945303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c016d9c-bb61-45dd-8690-f705e953b0e6 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:38.945571 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c016d9c-bb61-45dd-8690-f705e953b0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:38.945840 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c016d9c-bb61-45dd-8690-f705e953b0e6 None admin] PATH_INFO: `/v3/projects/785791bda4044663bfe6cd25439a8358` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:38.946950 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c016d9c-bb61-45dd-8690-f705e953b0e6 None admin] RBAC: Authorizing `identity:get_project(project_id=785791bda4044663bfe6cd25439a8358)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:38.948383 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c016d9c-bb61-45dd-8690-f705e953b0e6 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:38.950201 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 223/452] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:23:38 2022] GET /identity/v3/projects/785791bda4044663bfe6cd25439a8358 => generated 344 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:38.956014 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c016d9c-bb61-45dd-8690-f705e953b0e6 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:38.962769 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:38.979197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5136aa-7d32-45fb-97de-136d63b28cab None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:39.013224 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb08ca9-e4a6-4450-b747-8e5d5b1d9703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:39.013778 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb08ca9-e4a6-4450-b747-8e5d5b1d9703 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:39.014060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb08ca9-e4a6-4450-b747-8e5d5b1d9703 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:39.014335 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb08ca9-e4a6-4450-b747-8e5d5b1d9703 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:39.015103 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb08ca9-e4a6-4450-b747-8e5d5b1d9703 None admin] RBAC: Adding query filter params (group.id=c071340c0d454c1488b9dccd5561b053, role.id=152f4082bb67479ebaeb08c2fa683399, scope.project.id=3ab93f396edd405c96c6effd6d5edb41) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:39.015494 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb08ca9-e4a6-4450-b747-8e5d5b1d9703 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:39.016925 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb08ca9-e4a6-4450-b747-8e5d5b1d9703 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:39.022552 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 230/453] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Aug 8 08:23:38 2022] GET /identity/v3/role_assignments?group.id=c071340c0d454c1488b9dccd5561b053&scope.project.id=3ab93f396edd405c96c6effd6d5edb41&role.id=152f4082bb67479ebaeb08c2fa683399 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:39.059521 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c016d9c-bb61-45dd-8690-f705e953b0e6 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:39.090593 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:39.096423 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda75514-9acd-4f5b-9a48-3a59f76c5bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:39.097058 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda75514-9acd-4f5b-9a48-3a59f76c5bda None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:39.097355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda75514-9acd-4f5b-9a48-3a59f76c5bda None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:39.097629 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda75514-9acd-4f5b-9a48-3a59f76c5bda None admin] PATH_INFO: `/v3/groups/9fa2c82d9d994f3f89e691f6b706993f` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:39.098788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda75514-9acd-4f5b-9a48-3a59f76c5bda None admin] RBAC: Authorizing `identity:get_group(group_id=9fa2c82d9d994f3f89e691f6b706993f)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:39.100242 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda75514-9acd-4f5b-9a48-3a59f76c5bda None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:39.101976 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 224/454] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 08:23:38 2022] GET /identity/v3/groups/9fa2c82d9d994f3f89e691f6b706993f => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:39.107886 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb08ca9-e4a6-4450-b747-8e5d5b1d9703 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:39.211757 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb08ca9-e4a6-4450-b747-8e5d5b1d9703 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:39.245484 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e0323-afb2-4533-819c-13ca7675832a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:39.246101 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e0323-afb2-4533-819c-13ca7675832a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:39.246383 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e0323-afb2-4533-819c-13ca7675832a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:39.246656 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e0323-afb2-4533-819c-13ca7675832a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:39.247378 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e0323-afb2-4533-819c-13ca7675832a None admin] RBAC: Adding query filter params (group.id=9fa2c82d9d994f3f89e691f6b706993f, role.id=c1e514a4b5424f959b069f36cbfd8147, scope.project.id=785791bda4044663bfe6cd25439a8358) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:39.247685 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e0323-afb2-4533-819c-13ca7675832a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:39.249295 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e0323-afb2-4533-819c-13ca7675832a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:39.255471 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 231/455] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Aug 8 08:23:39 2022] GET /identity/v3/role_assignments?group.id=9fa2c82d9d994f3f89e691f6b706993f&scope.project.id=785791bda4044663bfe6cd25439a8358&role.id=c1e514a4b5424f959b069f36cbfd8147 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:41.831683 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88160fb8-62e7-4f2a-b5b2-2c4e4ec74866 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:41.832178 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88160fb8-62e7-4f2a-b5b2-2c4e4ec74866 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:41.832178 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88160fb8-62e7-4f2a-b5b2-2c4e4ec74866 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:41.833759 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 225/456] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:41.839301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ccc63-282a-4b49-81fb-ee74aa621b26 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:41.840053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ccc63-282a-4b49-81fb-ee74aa621b26 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:41.840471 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ccc63-282a-4b49-81fb-ee74aa621b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:41.867359 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-002ccc63-282a-4b49-81fb-ee74aa621b26 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:41.868796 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-002ccc63-282a-4b49-81fb-ee74aa621b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:41.879860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:41.955494 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 232/457] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:41.996951 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88160fb8-62e7-4f2a-b5b2-2c4e4ec74866 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:42.135477 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88160fb8-62e7-4f2a-b5b2-2c4e4ec74866 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:42.140400 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f79647b-93f7-4295-a96c-2bde67e4268d None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:42.140683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f79647b-93f7-4295-a96c-2bde67e4268d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:42.140955 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f79647b-93f7-4295-a96c-2bde67e4268d None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:42.142321 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 233/458] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:42.155127 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a99c4e-6e14-4903-bc59-b4842624389f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:42.155940 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a99c4e-6e14-4903-bc59-b4842624389f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:42.156214 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a99c4e-6e14-4903-bc59-b4842624389f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:42.175135 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-43a99c4e-6e14-4903-bc59-b4842624389f None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:42.176615 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-43a99c4e-6e14-4903-bc59-b4842624389f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:42.184433 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35061d-948b-4449-9ec4-66754de6d925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:42.185184 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35061d-948b-4449-9ec4-66754de6d925 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:42.185363 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35061d-948b-4449-9ec4-66754de6d925 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:42.185643 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35061d-948b-4449-9ec4-66754de6d925 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:42.198278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b35061d-948b-4449-9ec4-66754de6d925 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:42.200157 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b35061d-948b-4449-9ec4-66754de6d925 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:42.219613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-9b35061d-948b-4449-9ec4-66754de6d925 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:42.219613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:42.219613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:42.219613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:42.219613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:42.219613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:42.219613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:42.219613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:42.219613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:42.219613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:42.219613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:42.219613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:42.219613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:23:42.219613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:23:42.219613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:23:42.219613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:23:42.220564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:42.220564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:42.220564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:42.220564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:42.220564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:42.220564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:42.220564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:42.220564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:42.220564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:42.220564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:42.220564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:42.220564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:42.220564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:42.220564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:42.220564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:42.220564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:42.220564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:23:42.221429 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:23:42.221429 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:42.221429 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:42.221429 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 226/459] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:42.222355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af404a-1a04-456e-ae60-a340f83073a7 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:42.222636 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af404a-1a04-456e-ae60-a340f83073a7 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:42.222905 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af404a-1a04-456e-ae60-a340f83073a7 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:42.224659 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 227/460] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:42.229854 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af404a-1a04-456e-ae60-a340f83073a7 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:42.261280 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 234/461] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:23:42.262967 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1440d2-2088-4ad2-a0e3-3768499feebf None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:42.263441 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1440d2-2088-4ad2-a0e3-3768499feebf None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:42.263904 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1440d2-2088-4ad2-a0e3-3768499feebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:42.298755 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-cf1440d2-2088-4ad2-a0e3-3768499feebf None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:42.300233 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-cf1440d2-2088-4ad2-a0e3-3768499feebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:42.306258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:42.363780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af404a-1a04-456e-ae60-a340f83073a7 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:42.381068 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 235/462] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:42.382101 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1440d2-2088-4ad2-a0e3-3768499feebf None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:42.384347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:42.417453 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901679af-c090-49ae-b9d3-be5549bb88a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:42.418262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901679af-c090-49ae-b9d3-be5549bb88a5 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:42.418543 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901679af-c090-49ae-b9d3-be5549bb88a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:42.418815 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901679af-c090-49ae-b9d3-be5549bb88a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:42.419471 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901679af-c090-49ae-b9d3-be5549bb88a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:42.419775 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901679af-c090-49ae-b9d3-be5549bb88a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:42.422099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901679af-c090-49ae-b9d3-be5549bb88a5 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:42.434981 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 228/463] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:42.436215 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a295f7-1434-440d-b169-d222e848ebbd None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:42.437131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a295f7-1434-440d-b169-d222e848ebbd None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:42.437446 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a295f7-1434-440d-b169-d222e848ebbd None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:42.441281 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 229/464] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:42.442234 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a295f7-1434-440d-b169-d222e848ebbd None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:42.500249 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1440d2-2088-4ad2-a0e3-3768499feebf None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:42.533884 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a398b3-20c7-4a6c-b948-0231f8e4db88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:42.534525 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a398b3-20c7-4a6c-b948-0231f8e4db88 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:42.534800 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a398b3-20c7-4a6c-b948-0231f8e4db88 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:42.535069 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a398b3-20c7-4a6c-b948-0231f8e4db88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:42.546384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a398b3-20c7-4a6c-b948-0231f8e4db88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:42.547910 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a398b3-20c7-4a6c-b948-0231f8e4db88 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:42.551356 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a295f7-1434-440d-b169-d222e848ebbd None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:42.562766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:42.563828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-f7a398b3-20c7-4a6c-b948-0231f8e4db88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:42.563828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:42.563828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:42.563828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:42.563828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:42.563828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:42.563828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:42.563828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:42.563828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:42.563828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:42.563828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:42.563828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:42.563828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:23:42.563828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:23:42.563828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:23:42.563828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:23:42.564709 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:42.564709 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:42.564709 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:42.564709 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:42.564709 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:42.564709 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:42.564709 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:42.564709 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:42.564709 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:42.564709 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:42.564709 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:42.564709 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:42.564709 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:42.564709 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:42.564709 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:42.564709 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:42.564709 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:23:42.566159 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:23:42.566159 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:42.566159 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:42.566159 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 236/465] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:42.566159 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a398b3-20c7-4a6c-b948-0231f8e4db88 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:42.592036 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf060e-e73c-4c58-af03-ff04b7611935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:42.592937 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf060e-e73c-4c58-af03-ff04b7611935 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:42.593487 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf060e-e73c-4c58-af03-ff04b7611935 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:42.593915 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf060e-e73c-4c58-af03-ff04b7611935 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:42.605882 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf060e-e73c-4c58-af03-ff04b7611935 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:42.607501 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf060e-e73c-4c58-af03-ff04b7611935 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:42.623299 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-a3bf060e-e73c-4c58-af03-ff04b7611935 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:42.623299 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:42.623299 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:42.623299 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:42.623299 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:42.623299 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:42.623299 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:42.623299 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:42.623299 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:42.623299 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:42.623299 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:42.623299 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:42.623299 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:23:42.623299 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:23:42.623299 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:23:42.623299 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:23:42.624917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:42.624917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:42.624917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:42.624917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:42.624917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:42.624917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:42.624917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:42.624917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:42.624917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:42.624917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:42.624917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:42.624917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:42.624917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:42.624917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:42.624917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:42.624917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:42.624917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:23:42.625819 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:23:42.625819 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:42.625819 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:42.626001 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:42.626076 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 230/466] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:42.626436 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d8caa-15b4-48ca-9f35-f766e1382955 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:42.626539 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d8caa-15b4-48ca-9f35-f766e1382955 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:42.626735 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d8caa-15b4-48ca-9f35-f766e1382955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:42.645060 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-497d8caa-15b4-48ca-9f35-f766e1382955 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:42.646251 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-497d8caa-15b4-48ca-9f35-f766e1382955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:42.675171 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a398b3-20c7-4a6c-b948-0231f8e4db88 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:42.709795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee80d4-a3dc-498a-b1ba-d279756d9a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:42.710434 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee80d4-a3dc-498a-b1ba-d279756d9a80 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:42.710718 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee80d4-a3dc-498a-b1ba-d279756d9a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:42.710992 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee80d4-a3dc-498a-b1ba-d279756d9a80 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:42.711632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee80d4-a3dc-498a-b1ba-d279756d9a80 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:42.713073 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee80d4-a3dc-498a-b1ba-d279756d9a80 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:42.715149 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 231/467] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:42.716132 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d8caa-15b4-48ca-9f35-f766e1382955 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:42.813032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:42.821136 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d8caa-15b4-48ca-9f35-f766e1382955 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:42.850575 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 237/468] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 08:23:42 2022] POST /identity/v3/users => generated 246 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:23:42.851564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee80d4-a3dc-498a-b1ba-d279756d9a80 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:42.854967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f22debd-6181-4663-bc2f-62e2323b5179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:42.855700 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22debd-6181-4663-bc2f-62e2323b5179 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:42.856262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22debd-6181-4663-bc2f-62e2323b5179 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:42.856695 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22debd-6181-4663-bc2f-62e2323b5179 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:42.857919 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f22debd-6181-4663-bc2f-62e2323b5179 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:42.858221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f22debd-6181-4663-bc2f-62e2323b5179 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:42.861307 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f22debd-6181-4663-bc2f-62e2323b5179 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:42.873409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 232/469] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:42.874350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f22debd-6181-4663-bc2f-62e2323b5179 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:42.975764 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee80d4-a3dc-498a-b1ba-d279756d9a80 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:42.987691 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:42.990564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f22debd-6181-4663-bc2f-62e2323b5179 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:43.016378 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a916151-1785-4f27-90a4-f55dad08b344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:43.016782 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a916151-1785-4f27-90a4-f55dad08b344 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:43.017065 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a916151-1785-4f27-90a4-f55dad08b344 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:43.018224 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a916151-1785-4f27-90a4-f55dad08b344 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:43.018726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a916151-1785-4f27-90a4-f55dad08b344 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:43.019257 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a916151-1785-4f27-90a4-f55dad08b344 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:43.021278 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a916151-1785-4f27-90a4-f55dad08b344 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:43.030033 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e566e-cc67-4fbf-9fe4-d072cea2e7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:43.031415 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e566e-cc67-4fbf-9fe4-d072cea2e7c7 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:43.039134 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e566e-cc67-4fbf-9fe4-d072cea2e7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:43.039134 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e566e-cc67-4fbf-9fe4-d072cea2e7c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:43.039313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 238/470] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:43.039313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b811d30-220f-455b-8828-a3cd3fe38bd5 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:43.039313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b811d30-220f-455b-8828-a3cd3fe38bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:43.039313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b811d30-220f-455b-8828-a3cd3fe38bd5 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:43.039313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 239/471] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:43.039313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b811d30-220f-455b-8828-a3cd3fe38bd5 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:43.047396 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8e566e-cc67-4fbf-9fe4-d072cea2e7c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:43.049110 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8e566e-cc67-4fbf-9fe4-d072cea2e7c7 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:43.066373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-cf8e566e-cc67-4fbf-9fe4-d072cea2e7c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:43.066373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:43.066373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:43.066373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:43.066373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:43.066373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:43.066373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:43.066373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:43.066373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:43.066373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:43.066373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:43.066373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:43.066373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:23:43.066373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:23:43.066373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:23:43.066373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:23:43.067329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:43.067329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:43.067329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:43.067329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:43.067329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:43.067329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:43.067329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:43.067329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:43.067329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:43.067329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:43.067329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:43.067329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:43.067329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:43.067329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:43.067329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:43.067329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:43.067329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:23:43.068189 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:23:43.068189 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:43.068189 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:43.069987 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 233/472] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:43.070894 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e566e-cc67-4fbf-9fe4-d072cea2e7c7 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:43.075199 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:43.152921 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b811d30-220f-455b-8828-a3cd3fe38bd5 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:43.177932 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e566e-cc67-4fbf-9fe4-d072cea2e7c7 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:43.189979 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3711e431-45ab-497b-95b1-358345b4879b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:43.190594 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711e431-45ab-497b-95b1-358345b4879b None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:43.190986 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711e431-45ab-497b-95b1-358345b4879b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:43.191257 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711e431-45ab-497b-95b1-358345b4879b None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:43.192055 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3711e431-45ab-497b-95b1-358345b4879b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:43.193535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3711e431-45ab-497b-95b1-358345b4879b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:43.213472 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d344f3d-3f2c-47df-be91-9e34ef56b11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:43.216163 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d344f3d-3f2c-47df-be91-9e34ef56b11f None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:43.216163 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d344f3d-3f2c-47df-be91-9e34ef56b11f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:43.216163 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d344f3d-3f2c-47df-be91-9e34ef56b11f None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:43.216163 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d344f3d-3f2c-47df-be91-9e34ef56b11f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:43.216733 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d344f3d-3f2c-47df-be91-9e34ef56b11f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:43.256348 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 240/473] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 08:23:43 2022] POST /identity/v3/users => generated 248 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:23:43.257642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb464363-5d25-448d-9ffe-96bf9fe83cd5 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:43.257937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb464363-5d25-448d-9ffe-96bf9fe83cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:43.258186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb464363-5d25-448d-9ffe-96bf9fe83cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:43.278373 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-fb464363-5d25-448d-9ffe-96bf9fe83cd5 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:43.280899 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fb464363-5d25-448d-9ffe-96bf9fe83cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:43.281069 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 234/474] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 08:23:43 2022] POST /identity/v3/users => generated 248 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:23:43.281534 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d344f3d-3f2c-47df-be91-9e34ef56b11f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:43.321539 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:43.410652 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:43.432359 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 241/475] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:43.470175 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb464363-5d25-448d-9ffe-96bf9fe83cd5 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:43.471926 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d344f3d-3f2c-47df-be91-9e34ef56b11f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:43.520734 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42baef5f-bfdc-4ae2-b689-428b9e686d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:43.521300 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42baef5f-bfdc-4ae2-b689-428b9e686d5d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:43.521588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42baef5f-bfdc-4ae2-b689-428b9e686d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:43.521864 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42baef5f-bfdc-4ae2-b689-428b9e686d5d None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:43.522519 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42baef5f-bfdc-4ae2-b689-428b9e686d5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:43.522821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42baef5f-bfdc-4ae2-b689-428b9e686d5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:43.524624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42baef5f-bfdc-4ae2-b689-428b9e686d5d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:43.539500 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 235/476] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:43.546201 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42baef5f-bfdc-4ae2-b689-428b9e686d5d None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:43.606227 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb464363-5d25-448d-9ffe-96bf9fe83cd5 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:43.649289 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc2f0a5-bdc8-43e7-9b07-1e0981e4f478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:43.650411 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2f0a5-bdc8-43e7-9b07-1e0981e4f478 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:43.651219 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2f0a5-bdc8-43e7-9b07-1e0981e4f478 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:43.651642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2f0a5-bdc8-43e7-9b07-1e0981e4f478 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:43.653172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc2f0a5-bdc8-43e7-9b07-1e0981e4f478 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:43.655913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc2f0a5-bdc8-43e7-9b07-1e0981e4f478 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:43.663212 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42baef5f-bfdc-4ae2-b689-428b9e686d5d None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:43.685483 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.sql.core [None req-dfc2f0a5-bdc8-43e7-9b07-1e0981e4f478 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 08:23:43.685483 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 08 08:23:43.685483 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [parameters: {'id': 'ef684ced21a14bcea0c8c62096926391', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 08 08:23:43.685483 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80261) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 08:23:43.688440 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:43.702514 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60648ca4-9131-40ad-a743-e638eb700c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:43.703359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60648ca4-9131-40ad-a743-e638eb700c33 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:43.703686 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60648ca4-9131-40ad-a743-e638eb700c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:43.703956 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60648ca4-9131-40ad-a743-e638eb700c33 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:43.704655 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60648ca4-9131-40ad-a743-e638eb700c33 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:43.706110 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60648ca4-9131-40ad-a743-e638eb700c33 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:43.709016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-dfc2f0a5-bdc8-43e7-9b07-1e0981e4f478 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 08 08:23:43.709016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:43.709016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 08:23:43.709016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 08:23:43.709016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 08:23:43.709016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 08:23:43.709016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 08:23:43.709016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 08:23:43.709016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 08:23:43.709016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application conn.query(q) Aug 08 08:23:43.709016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 08:23:43.709016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 08:23:43.709016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 08:23:43.709016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application result.read() Aug 08 08:23:43.709016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 08:23:43.720900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 08:23:43.720900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 08:23:43.720900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 08:23:43.720900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 08:23:43.720900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 08:23:43.720900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 08:23:43.720900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 08:23:43.720900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 08:23:43.720900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:43.720900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 08:23:43.720900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:43.720900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:43.720900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 08:23:43.720900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 08:23:43.720900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 08 08:23:43.720900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return ref.to_dict() Aug 08 08:23:43.720900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 08:23:43.720900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application next(self.gen) Aug 08 08:23:43.721819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 08:23:43.721819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application yield resource Aug 08 08:23:43.721819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 08:23:43.721819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application next(self.gen) Aug 08 08:23:43.721819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 08:23:43.721819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 08:23:43.721819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 08:23:43.721819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 08:23:43.721819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 08:23:43.721819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise self.value Aug 08 08:23:43.721819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 08:23:43.721819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 08:23:43.721819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 08:23:43.721819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application session.commit() Aug 08 08:23:43.721819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 08:23:43.721819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 08:23:43.721819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 08:23:43.722706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 08:23:43.722706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 08:23:43.722706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self.session.flush() Aug 08 08:23:43.722706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 08:23:43.722706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 08:23:43.722706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 08:23:43.722706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 08:23:43.722706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 08:23:43.722706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application compat.raise_( Aug 08 08:23:43.722706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 08:23:43.722706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception Aug 08 08:23:43.722706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 08:23:43.722706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 08:23:43.722706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 08:23:43.722706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 08:23:43.722706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 08:23:43.722706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 08:23:43.723551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 08:23:43.723551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 08:23:43.723551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 08 08:23:43.723551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 08 08:23:43.723551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 08:23:43.723551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 08:23:43.723551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 08:23:43.723551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 08:23:43.723551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 08:23:43.723551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 08:23:43.723551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 08:23:43.723551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 08:23:43.723551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 08:23:43.723551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 08:23:43.723551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 08:23:43.723551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception Aug 08 08:23:43.724377 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 08:23:43.724377 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 08:23:43.724377 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 08:23:43.724377 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 08:23:43.724377 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 08:23:43.724377 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 08:23:43.724377 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 08:23:43.724377 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application conn.query(q) Aug 08 08:23:43.724377 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 08:23:43.724377 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 08:23:43.724377 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 08:23:43.724377 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application result.read() Aug 08 08:23:43.724377 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 08:23:43.724377 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 08:23:43.724377 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 08:23:43.724377 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 08:23:43.725386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 08:23:43.725386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 08:23:43.725386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 08:23:43.725386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 08:23:43.725386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 08:23:43.725386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 08 08:23:43.725386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [parameters: {'id': 'ef684ced21a14bcea0c8c62096926391', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 08 08:23:43.725386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 08:23:43.725386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:43.725386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:23:43.725386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:43.725386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:43.725386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:43.725386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:43.725386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:43.725386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:43.726289 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:43.726289 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:43.726289 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:43.726289 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:43.726289 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:43.726289 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:43.726289 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 08 08:23:43.726289 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 08 08:23:43.726289 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:43.726289 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:43.726289 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 08 08:23:43.726289 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 08 08:23:43.726289 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 08:23:43.726289 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 08:23:43.726289 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 08 08:23:43.726289 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:43.727874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 242/477] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Aug 8 08:23:43 2022] POST /identity/v3/roles => generated 144 bytes in 241 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:43.727874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc2f0a5-bdc8-43e7-9b07-1e0981e4f478 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:43.753944 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:43.788290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 236/478] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 08:23:43 2022] POST /identity/v3/users => generated 249 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:23:43.848738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc2f0a5-bdc8-43e7-9b07-1e0981e4f478 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:43.897511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762db80a-2e49-4a80-b22d-679b8f81a670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:43.898311 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762db80a-2e49-4a80-b22d-679b8f81a670 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:43.898467 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762db80a-2e49-4a80-b22d-679b8f81a670 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:43.898732 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762db80a-2e49-4a80-b22d-679b8f81a670 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:43.921722 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762db80a-2e49-4a80-b22d-679b8f81a670 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:43.930091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-762db80a-2e49-4a80-b22d-679b8f81a670 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 08:23:43.930091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:43.930091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:43.930091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:43.930091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:43.930091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:43.930091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:43.930091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:43.930091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:43.930091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:43.930091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:43.930091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:43.930091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:43.930091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:43.930091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:43.930091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:43.932425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:43.932425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:43.932425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:43.932425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:43.932425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:43.932425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:43.932425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:43.932425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:43.932425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:43.932425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:43.932425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:43.932425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:43.932425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:43.932425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:43.932425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:43.932425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:43.934424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:43.934424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:43.934424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:43.934424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:43.934424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:43.934424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:43.934424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:43.934424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:43.934424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:43.934424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:43.934424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 08:23:43.934424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:43.934424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 243/479] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 08:23:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:43.941088 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60648ca4-9131-40ad-a743-e638eb700c33 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:44.051856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60648ca4-9131-40ad-a743-e638eb700c33 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:44.087214 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d8654-74e6-4643-84be-509efb399319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:44.087822 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d8654-74e6-4643-84be-509efb399319 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:44.088100 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d8654-74e6-4643-84be-509efb399319 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:44.088367 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d8654-74e6-4643-84be-509efb399319 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:44.089028 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d8654-74e6-4643-84be-509efb399319 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:44.089367 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d8654-74e6-4643-84be-509efb399319 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:44.090723 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d8654-74e6-4643-84be-509efb399319 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:44.099426 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 237/480] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:23:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:45.214188 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b224df8-6561-41ad-8d3d-d59da6118904 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:45.214702 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b224df8-6561-41ad-8d3d-d59da6118904 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:45.214702 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b224df8-6561-41ad-8d3d-d59da6118904 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:45.216285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 244/481] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:45.222357 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff81193-95d6-40c1-bca9-7ee25c103325 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:45.222755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff81193-95d6-40c1-bca9-7ee25c103325 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:45.222955 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff81193-95d6-40c1-bca9-7ee25c103325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:45.250438 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-1ff81193-95d6-40c1-bca9-7ee25c103325 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:45.251615 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1ff81193-95d6-40c1-bca9-7ee25c103325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:45.281374 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:45.319482 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 238/482] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:23:45.342330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b224df8-6561-41ad-8d3d-d59da6118904 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:45.463272 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:45.493918 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b224df8-6561-41ad-8d3d-d59da6118904 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:45.541870 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03600a35-1454-440a-bfc6-345dfeda42dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:45.542502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03600a35-1454-440a-bfc6-345dfeda42dd None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:45.542782 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03600a35-1454-440a-bfc6-345dfeda42dd None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:45.543053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03600a35-1454-440a-bfc6-345dfeda42dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:45.562561 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03600a35-1454-440a-bfc6-345dfeda42dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:45.567420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-03600a35-1454-440a-bfc6-345dfeda42dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:23:45.567420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:45.567420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:45.567420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:45.567420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:45.567420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:45.567420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:45.567420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:45.567420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:45.567420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:45.567420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:45.567420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:45.567420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:45.567420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:45.567420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:45.567420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:45.568317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:45.568317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:45.568317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:45.568317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:45.568317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:45.568317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:45.568317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:45.568317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:45.568317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:45.568317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:45.568317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:45.568317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:45.568317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:45.568317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:45.568317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:45.568317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:45.569104 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:45.569104 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:45.569104 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:45.569104 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:45.569104 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:45.569104 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:45.569104 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:45.569104 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:45.569104 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:45.569104 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:45.569104 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:23:45.569104 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:45.569104 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 245/483] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 08:23:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:45.578301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff81193-95d6-40c1-bca9-7ee25c103325 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:45.715354 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff81193-95d6-40c1-bca9-7ee25c103325 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:45.763857 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf68d5-0b83-40fd-9f99-9bba54743b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:45.764387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf68d5-0b83-40fd-9f99-9bba54743b39 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:45.764726 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf68d5-0b83-40fd-9f99-9bba54743b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:45.764995 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf68d5-0b83-40fd-9f99-9bba54743b39 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:45.765686 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf68d5-0b83-40fd-9f99-9bba54743b39 None admin] RBAC: Adding query filter params (name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:45.765985 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf68d5-0b83-40fd-9f99-9bba54743b39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:45.767326 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf68d5-0b83-40fd-9f99-9bba54743b39 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:45.775725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 239/484] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 08:23:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:45.781782 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03600a35-1454-440a-bfc6-345dfeda42dd None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:45.895354 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbd80b-71da-480e-9b8e-44033968636d None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:45.895669 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbd80b-71da-480e-9b8e-44033968636d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:45.895896 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbd80b-71da-480e-9b8e-44033968636d None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:45.897474 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 240/485] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:45.902376 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93f1b3-3f58-435a-8b47-f6561b7f7a89 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:45.902651 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93f1b3-3f58-435a-8b47-f6561b7f7a89 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:45.902923 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93f1b3-3f58-435a-8b47-f6561b7f7a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:45.926044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-0a93f1b3-3f58-435a-8b47-f6561b7f7a89 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:45.927211 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0a93f1b3-3f58-435a-8b47-f6561b7f7a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:45.933642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03600a35-1454-440a-bfc6-345dfeda42dd None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:45.981512 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:45.995326 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dffbf2-070d-4e86-b0df-7998563740c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:45.995963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dffbf2-070d-4e86-b0df-7998563740c7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:45.996240 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dffbf2-070d-4e86-b0df-7998563740c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:45.996511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dffbf2-070d-4e86-b0df-7998563740c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:46.015372 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:46.018464 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dffbf2-070d-4e86-b0df-7998563740c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:46.019950 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dffbf2-070d-4e86-b0df-7998563740c7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:46.049366 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 241/486] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:23:46.050075 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-71dffbf2-070d-4e86-b0df-7998563740c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:46.050075 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:46.050075 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:46.050075 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:46.050075 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:46.050075 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:46.050075 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:46.050075 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:46.050075 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:46.050075 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:46.050075 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:46.050075 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:46.050075 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:23:46.050075 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:23:46.050075 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:23:46.050075 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:23:46.050973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:46.050973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:46.050973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:46.050973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:46.050973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:46.050973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:46.050973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:46.050973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:46.050973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:46.050973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:46.050973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:46.050973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:46.050973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:46.050973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:46.050973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:46.050973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:46.050973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:23:46.051795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:23:46.051795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:46.051795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:46.051795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 246/487] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:46.057930 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93f1b3-3f58-435a-8b47-f6561b7f7a89 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:46.081019 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dffbf2-070d-4e86-b0df-7998563740c7 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:46.185048 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93f1b3-3f58-435a-8b47-f6561b7f7a89 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:46.199413 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dffbf2-070d-4e86-b0df-7998563740c7 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:46.235211 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6ccb2-f547-4e63-9a4a-3b7909661490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:46.235757 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a60e0-08b6-4d4d-b3e9-137d55a747ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:46.236129 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a60e0-08b6-4d4d-b3e9-137d55a747ee None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:46.236563 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a60e0-08b6-4d4d-b3e9-137d55a747ee None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:46.236908 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a60e0-08b6-4d4d-b3e9-137d55a747ee None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:46.237390 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6ccb2-f547-4e63-9a4a-3b7909661490 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:46.237894 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6ccb2-f547-4e63-9a4a-3b7909661490 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:46.238410 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6ccb2-f547-4e63-9a4a-3b7909661490 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:46.239574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe6ccb2-f547-4e63-9a4a-3b7909661490 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:46.240062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe6ccb2-f547-4e63-9a4a-3b7909661490 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:46.242081 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe6ccb2-f547-4e63-9a4a-3b7909661490 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:46.248802 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134a60e0-08b6-4d4d-b3e9-137d55a747ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:46.255110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-134a60e0-08b6-4d4d-b3e9-137d55a747ee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:23:46.255110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:46.255110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:46.255110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:46.255110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:46.255110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:46.255110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:46.255110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:46.255110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:46.255110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:46.255110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:46.255110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:46.255110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:46.255110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:46.255110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:46.255110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:46.255989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:46.255989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:46.255989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:46.255989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:46.255989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:46.255989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:46.255989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:46.255989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:46.255989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:46.255989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:46.255989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:46.255989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:46.255989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:46.255989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:46.255989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:46.255989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:46.256760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:46.256760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:46.256760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:46.256760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:46.256760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:46.256760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:46.256760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:46.256760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:46.256760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:46.256760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:46.256760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:23:46.256760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:46.263275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 247/489] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 08:23:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:23:46.263275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d458fb9-0cb8-458c-8eeb-cfc26cf911a8 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:46.263275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d458fb9-0cb8-458c-8eeb-cfc26cf911a8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:46.263275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d458fb9-0cb8-458c-8eeb-cfc26cf911a8 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:46.263753 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 242/489] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:23:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:46.264380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 248/490] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:46.265081 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6ccb2-f547-4e63-9a4a-3b7909661490 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:46.265293 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d458fb9-0cb8-458c-8eeb-cfc26cf911a8 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:46.294070 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:46.334693 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:46.395488 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d458fb9-0cb8-458c-8eeb-cfc26cf911a8 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:46.398421 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6ccb2-f547-4e63-9a4a-3b7909661490 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:46.437743 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b480c8-dc50-4e37-9aaa-c8d9290dedd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:46.438397 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b480c8-dc50-4e37-9aaa-c8d9290dedd0 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:46.438649 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b480c8-dc50-4e37-9aaa-c8d9290dedd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:46.438942 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b480c8-dc50-4e37-9aaa-c8d9290dedd0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:46.444093 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542e562-199e-4651-8ae9-10cdf2aa48bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:46.445002 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542e562-199e-4651-8ae9-10cdf2aa48bd None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:46.445892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542e562-199e-4651-8ae9-10cdf2aa48bd None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:46.446658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542e562-199e-4651-8ae9-10cdf2aa48bd None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:46.447807 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8542e562-199e-4651-8ae9-10cdf2aa48bd None admin] RBAC: Adding query filter params (name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:46.448319 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8542e562-199e-4651-8ae9-10cdf2aa48bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:46.449531 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b480c8-dc50-4e37-9aaa-c8d9290dedd0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:46.450149 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8542e562-199e-4651-8ae9-10cdf2aa48bd None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:46.451063 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b480c8-dc50-4e37-9aaa-c8d9290dedd0 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:46.461398 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 243/491] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 08:23:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:46.463392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad6ca6-1a0a-4540-9677-e1ed4d79fc4f None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:46.463878 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad6ca6-1a0a-4540-9677-e1ed4d79fc4f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:46.464674 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad6ca6-1a0a-4540-9677-e1ed4d79fc4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:46.466978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-45b480c8-dc50-4e37-9aaa-c8d9290dedd0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 08:23:46.466978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:46.466978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:46.466978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:46.466978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:46.466978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:46.466978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:46.466978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:46.466978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:46.466978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:46.466978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:46.466978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:46.466978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:23:46.466978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:23:46.466978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:23:46.466978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:23:46.468367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:46.468367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:46.468367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:23:46.468367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:23:46.468367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:23:46.468367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:23:46.468367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:46.468367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:46.468367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:46.468367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:46.468367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:46.468367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:46.468367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:46.468367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:46.468367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:46.468367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:46.468367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:46.468367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:46.469936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:46.469936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:46.469936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:23:46.469936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:23:46.469936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:23:46.469936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:23:46.469936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:23:46.469936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:23:46.469936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 08:23:46.469936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:46.473227 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 249/492] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Aug 8 08:23:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:46.474389 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b480c8-dc50-4e37-9aaa-c8d9290dedd0 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:46.492536 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-8bad6ca6-1a0a-4540-9677-e1ed4d79fc4f None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:46.494218 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8bad6ca6-1a0a-4540-9677-e1ed4d79fc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:46.574433 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 244/493] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:46.575813 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad6ca6-1a0a-4540-9677-e1ed4d79fc4f None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:46.597402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:46.605972 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b480c8-dc50-4e37-9aaa-c8d9290dedd0 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:46.622260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:46.649109 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c6777-e0e0-4fec-9a71-5113fc0919fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:46.649777 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c6777-e0e0-4fec-9a71-5113fc0919fa None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:46.650064 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c6777-e0e0-4fec-9a71-5113fc0919fa None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:46.650329 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c6777-e0e0-4fec-9a71-5113fc0919fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:46.663230 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026c6777-e0e0-4fec-9a71-5113fc0919fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:46.664722 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026c6777-e0e0-4fec-9a71-5113fc0919fa None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:46.683454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-026c6777-e0e0-4fec-9a71-5113fc0919fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:46.683454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:46.683454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:46.683454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:46.683454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:46.683454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:46.683454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:46.683454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:46.683454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:46.683454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:46.683454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:46.683454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:46.683454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:23:46.683454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:23:46.683454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:23:46.683454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:23:46.684433 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:46.684433 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:46.684433 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:46.684433 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:46.684433 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:46.684433 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:46.684433 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:46.684433 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:46.684433 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:46.684433 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:46.684433 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:46.684433 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:46.684433 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:46.684433 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:46.684433 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:46.684433 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:46.684433 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:23:46.685291 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:23:46.685291 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:46.685291 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:46.691497 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 250/494] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:46.693272 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317abddd-f589-4bd2-b130-7f7cc8a73930 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:46.693568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317abddd-f589-4bd2-b130-7f7cc8a73930 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:46.693845 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317abddd-f589-4bd2-b130-7f7cc8a73930 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:46.695277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 251/495] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:46.699952 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317abddd-f589-4bd2-b130-7f7cc8a73930 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:46.705381 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad6ca6-1a0a-4540-9677-e1ed4d79fc4f None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:46.746275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6b317-9d14-429a-8bec-a6605150be1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:46.747239 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6b317-9d14-429a-8bec-a6605150be1f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:46.747532 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6b317-9d14-429a-8bec-a6605150be1f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:46.747793 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6b317-9d14-429a-8bec-a6605150be1f None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:46.748542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6b317-9d14-429a-8bec-a6605150be1f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:46.748852 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6b317-9d14-429a-8bec-a6605150be1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:46.751078 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6b317-9d14-429a-8bec-a6605150be1f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:46.792218 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 245/496] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Aug 8 08:23:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:46.793548 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6b317-9d14-429a-8bec-a6605150be1f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:46.820009 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317abddd-f589-4bd2-b130-7f7cc8a73930 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:46.855302 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc87741-cdc4-4dc7-b622-b02ec52d4c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:46.855852 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc87741-cdc4-4dc7-b622-b02ec52d4c74 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:46.856128 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc87741-cdc4-4dc7-b622-b02ec52d4c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:46.856399 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc87741-cdc4-4dc7-b622-b02ec52d4c74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:46.868155 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc87741-cdc4-4dc7-b622-b02ec52d4c74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:46.870000 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:46.875417 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-2fc87741-cdc4-4dc7-b622-b02ec52d4c74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:23:46.875417 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:46.875417 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:46.875417 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:46.875417 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:46.875417 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:46.875417 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:46.875417 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:46.875417 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:46.875417 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:46.875417 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:46.875417 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:46.875417 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:46.875417 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:46.875417 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:46.875417 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:46.888010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:46.888010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:46.888010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:46.888010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:46.888010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:46.888010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:46.888010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:46.888010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:46.888010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:46.888010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:46.888010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:46.888010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:46.888010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:46.888010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:46.888010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:46.888010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:46.888798 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:46.888798 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:46.888798 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:46.888798 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:46.888798 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:46.888798 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:46.888798 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:46.888798 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:46.888798 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:46.888798 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:46.888798 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:23:46.888798 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:46.888798 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 252/497] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 08:23:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:46.888798 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee788b-3f35-4626-b5a9-9baffbfd475a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:46.889628 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee788b-3f35-4626-b5a9-9baffbfd475a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:46.889628 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee788b-3f35-4626-b5a9-9baffbfd475a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:46.900057 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-87ee788b-3f35-4626-b5a9-9baffbfd475a None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:46.901300 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-87ee788b-3f35-4626-b5a9-9baffbfd475a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:46.901482 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6b317-9d14-429a-8bec-a6605150be1f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:46.919721 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:46.942959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08046c-2359-408e-ae54-bf59f720b28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:46.943576 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08046c-2359-408e-ae54-bf59f720b28f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:46.943857 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08046c-2359-408e-ae54-bf59f720b28f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:46.944149 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08046c-2359-408e-ae54-bf59f720b28f None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:46.944826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08046c-2359-408e-ae54-bf59f720b28f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:46.945108 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08046c-2359-408e-ae54-bf59f720b28f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:46.946564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08046c-2359-408e-ae54-bf59f720b28f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:46.958904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 246/498] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:23:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:46.959966 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56052ad5-ed5b-4115-a68f-6cc5374a7136 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:46.960233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56052ad5-ed5b-4115-a68f-6cc5374a7136 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:46.960506 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56052ad5-ed5b-4115-a68f-6cc5374a7136 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:46.961741 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 247/499] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:46.962448 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56052ad5-ed5b-4115-a68f-6cc5374a7136 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:46.976717 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 253/500] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:46.977699 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee788b-3f35-4626-b5a9-9baffbfd475a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:47.073521 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56052ad5-ed5b-4115-a68f-6cc5374a7136 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:47.079701 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee788b-3f35-4626-b5a9-9baffbfd475a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:47.106644 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:47.113078 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb427f29-5201-423d-a98f-329abe21a4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:47.113714 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb427f29-5201-423d-a98f-329abe21a4da None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:47.113990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb427f29-5201-423d-a98f-329abe21a4da None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:47.114255 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb427f29-5201-423d-a98f-329abe21a4da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:47.114522 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0bbc32-a1c4-44f6-9ce4-aba76f019062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:47.115220 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bbc32-a1c4-44f6-9ce4-aba76f019062 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:47.115410 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bbc32-a1c4-44f6-9ce4-aba76f019062 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:47.115682 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0bbc32-a1c4-44f6-9ce4-aba76f019062 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:47.116351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0bbc32-a1c4-44f6-9ce4-aba76f019062 None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:47.116650 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0bbc32-a1c4-44f6-9ce4-aba76f019062 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:47.118063 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0bbc32-a1c4-44f6-9ce4-aba76f019062 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:47.126892 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 254/501] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 08:23:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:47.131896 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0bbc32-a1c4-44f6-9ce4-aba76f019062 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:47.146448 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:47.189105 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb427f29-5201-423d-a98f-329abe21a4da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:47.190684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb427f29-5201-423d-a98f-329abe21a4da None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:47.229224 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0bbc32-a1c4-44f6-9ce4-aba76f019062 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:47.263227 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f2ff99-4e9a-470c-b853-828d3bb93ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:47.263852 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2ff99-4e9a-470c-b853-828d3bb93ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:47.264107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2ff99-4e9a-470c-b853-828d3bb93ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:47.264374 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2ff99-4e9a-470c-b853-828d3bb93ee7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:47.272750 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f2ff99-4e9a-470c-b853-828d3bb93ee7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:47.274229 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f2ff99-4e9a-470c-b853-828d3bb93ee7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:47.285708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-e3f2ff99-4e9a-470c-b853-828d3bb93ee7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 08:23:47.285708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:47.285708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:47.285708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:47.285708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:47.285708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:47.285708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:47.285708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:47.285708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:47.285708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:47.285708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:47.285708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:47.285708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:23:47.285708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:23:47.285708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:23:47.285708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:23:47.286626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:47.286626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:47.286626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:23:47.286626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:23:47.286626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:23:47.286626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:23:47.286626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:47.286626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:47.286626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:47.286626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:47.286626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:47.286626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:47.286626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:47.286626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:47.286626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:47.286626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:47.286626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:47.286626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:47.287519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:47.287519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:47.287519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:23:47.287519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:23:47.287519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:23:47.287519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:23:47.287519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:23:47.287519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:23:47.287519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 08:23:47.287519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:47.287519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 255/502] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:47.288176 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183fa71-9725-49ad-a0e2-4021f10f8ee7 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:47.288286 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183fa71-9725-49ad-a0e2-4021f10f8ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:47.288556 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183fa71-9725-49ad-a0e2-4021f10f8ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:47.292963 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 248/503] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:23:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:23:47.294322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb427f29-5201-423d-a98f-329abe21a4da None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:47.309134 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-2183fa71-9725-49ad-a0e2-4021f10f8ee7 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:47.310322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2183fa71-9725-49ad-a0e2-4021f10f8ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:47.323437 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:47.378688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 256/504] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:47.379330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2183fa71-9725-49ad-a0e2-4021f10f8ee7 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:47.393737 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:47.401285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb427f29-5201-423d-a98f-329abe21a4da None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:47.437889 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340da3f-1d04-4888-88d4-c4fd3f783436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:47.438514 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340da3f-1d04-4888-88d4-c4fd3f783436 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:47.438792 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340da3f-1d04-4888-88d4-c4fd3f783436 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:47.439065 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340da3f-1d04-4888-88d4-c4fd3f783436 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:47.448963 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2340da3f-1d04-4888-88d4-c4fd3f783436 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:47.453658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-2340da3f-1d04-4888-88d4-c4fd3f783436 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:23:47.453658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:47.453658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:47.453658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:47.453658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:47.453658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:47.453658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:47.453658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:47.453658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:47.453658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:47.453658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:47.453658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:47.453658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:47.453658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:47.453658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:47.453658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:47.454563 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:47.454563 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:47.454563 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:47.454563 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:47.454563 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:47.454563 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:47.454563 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:47.454563 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:47.454563 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:47.454563 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:47.454563 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:47.454563 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:47.454563 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:47.454563 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:47.454563 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:47.454563 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:47.455352 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:47.455352 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:47.455352 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:47.455352 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:47.455352 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:47.455352 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:47.455352 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:47.455352 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:47.455352 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:47.455352 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:47.455352 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:23:47.455352 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:47.455352 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 249/505] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 08:23:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:47.456048 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340da3f-1d04-4888-88d4-c4fd3f783436 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:47.487372 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2183fa71-9725-49ad-a0e2-4021f10f8ee7 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:47.521341 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a7c30-b387-4bec-8f94-9aea4e2820e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:47.521975 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a7c30-b387-4bec-8f94-9aea4e2820e0 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:47.522254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a7c30-b387-4bec-8f94-9aea4e2820e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:47.522523 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a7c30-b387-4bec-8f94-9aea4e2820e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:47.533003 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58a7c30-b387-4bec-8f94-9aea4e2820e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:47.534562 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58a7c30-b387-4bec-8f94-9aea4e2820e0 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:47.544197 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:47.550021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-f58a7c30-b387-4bec-8f94-9aea4e2820e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:47.550021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:47.550021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:47.550021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:47.550021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:47.550021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:47.550021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:47.550021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:47.550021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:47.550021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:47.550021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:47.550021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:47.550021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:23:47.550021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:23:47.550021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:23:47.550021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:23:47.556265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:47.556265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:47.556265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:47.556265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:47.556265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:47.556265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:47.556265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:47.556265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:47.556265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:47.556265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:47.556265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:47.556265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:47.556265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:47.556265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:47.556265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:47.556265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:47.556265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:23:47.557343 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:23:47.557343 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:47.557343 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:47.557343 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 257/506] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:47.557343 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a7c30-b387-4bec-8f94-9aea4e2820e0 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:47.557672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340da3f-1d04-4888-88d4-c4fd3f783436 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:47.590313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db56bd5-0a0a-4bb5-aa32-a3d12b1ee756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:47.590904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db56bd5-0a0a-4bb5-aa32-a3d12b1ee756 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:47.591180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db56bd5-0a0a-4bb5-aa32-a3d12b1ee756 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:47.591450 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db56bd5-0a0a-4bb5-aa32-a3d12b1ee756 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:47.592172 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db56bd5-0a0a-4bb5-aa32-a3d12b1ee756 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:47.592472 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db56bd5-0a0a-4bb5-aa32-a3d12b1ee756 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:47.594443 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db56bd5-0a0a-4bb5-aa32-a3d12b1ee756 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:47.611460 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 250/507] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Aug 8 08:23:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:47.612242 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db56bd5-0a0a-4bb5-aa32-a3d12b1ee756 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:47.613112 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:47.655916 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a7c30-b387-4bec-8f94-9aea4e2820e0 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:47.691094 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d0a28-a3d8-4ee5-9a4b-6d31cae4f29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:47.691714 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d0a28-a3d8-4ee5-9a4b-6d31cae4f29b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:47.691990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d0a28-a3d8-4ee5-9a4b-6d31cae4f29b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:47.692256 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d0a28-a3d8-4ee5-9a4b-6d31cae4f29b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:47.702518 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d0a28-a3d8-4ee5-9a4b-6d31cae4f29b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:47.704049 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d0a28-a3d8-4ee5-9a4b-6d31cae4f29b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:47.710475 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db56bd5-0a0a-4bb5-aa32-a3d12b1ee756 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:47.717783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-6c7d0a28-a3d8-4ee5-9a4b-6d31cae4f29b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:47.717783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:47.717783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:47.717783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:47.717783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:47.717783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:47.717783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:47.717783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:47.717783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:47.717783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:47.717783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:47.717783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:47.717783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:23:47.717783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:23:47.717783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:23:47.717783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:23:47.718738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:47.718738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:47.718738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:47.718738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:47.718738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:47.718738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:47.718738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:47.718738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:47.718738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:47.718738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:47.718738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:47.718738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:47.718738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:47.718738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:47.718738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:47.718738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:47.718738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:23:47.719577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:23:47.719577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:47.719577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:47.719577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 258/508] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:47.720209 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d0a28-a3d8-4ee5-9a4b-6d31cae4f29b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:47.748431 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8924eca-2735-4cfd-98c6-a8e0ca7abf73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:47.749963 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8924eca-2735-4cfd-98c6-a8e0ca7abf73 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:47.750137 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8924eca-2735-4cfd-98c6-a8e0ca7abf73 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:47.750362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8924eca-2735-4cfd-98c6-a8e0ca7abf73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:47.760197 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8924eca-2735-4cfd-98c6-a8e0ca7abf73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:47.761677 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8924eca-2735-4cfd-98c6-a8e0ca7abf73 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:47.769979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:47.776412 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-a8924eca-2735-4cfd-98c6-a8e0ca7abf73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:47.776412 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:47.776412 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:47.776412 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:47.776412 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:47.776412 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:47.776412 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:47.776412 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:47.776412 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:47.776412 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:47.776412 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:47.776412 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:47.776412 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:23:47.776412 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:23:47.776412 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:23:47.776412 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:23:47.782701 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:47.782701 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:47.782701 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:47.782701 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:47.782701 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:47.782701 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:47.782701 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:47.782701 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:47.782701 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:47.782701 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:47.782701 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:47.782701 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:47.782701 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:47.782701 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:47.782701 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:47.782701 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:47.782701 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:23:47.783713 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:23:47.783713 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:47.783713 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:47.783713 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 251/509] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:47.783713 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8924eca-2735-4cfd-98c6-a8e0ca7abf73 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:47.823884 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d0a28-a3d8-4ee5-9a4b-6d31cae4f29b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:47.835577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:47.857792 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22dc20-4b1a-498a-80d1-529c1b774eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:47.858909 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22dc20-4b1a-498a-80d1-529c1b774eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:47.859367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22dc20-4b1a-498a-80d1-529c1b774eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:47.859660 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22dc20-4b1a-498a-80d1-529c1b774eb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:47.860341 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22dc20-4b1a-498a-80d1-529c1b774eb3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:47.860971 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22dc20-4b1a-498a-80d1-529c1b774eb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:47.862845 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22dc20-4b1a-498a-80d1-529c1b774eb3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:47.870458 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 259/510] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 08:23:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:47.871225 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22dc20-4b1a-498a-80d1-529c1b774eb3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:47.883900 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8924eca-2735-4cfd-98c6-a8e0ca7abf73 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:47.917229 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18fc89-b753-4304-aa0e-99d1c422c19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:47.917916 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18fc89-b753-4304-aa0e-99d1c422c19d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:47.918195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18fc89-b753-4304-aa0e-99d1c422c19d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:47.918468 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18fc89-b753-4304-aa0e-99d1c422c19d None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:47.919109 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d18fc89-b753-4304-aa0e-99d1c422c19d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:47.919407 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d18fc89-b753-4304-aa0e-99d1c422c19d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:47.921304 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d18fc89-b753-4304-aa0e-99d1c422c19d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:47.932946 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 252/511] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:23:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:47.933780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18fc89-b753-4304-aa0e-99d1c422c19d None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:47.974326 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22dc20-4b1a-498a-80d1-529c1b774eb3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:48.008022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e864b1e-8254-4c6c-a3f3-813b86c98d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:48.008633 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e864b1e-8254-4c6c-a3f3-813b86c98d58 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:48.008915 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e864b1e-8254-4c6c-a3f3-813b86c98d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:48.009216 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e864b1e-8254-4c6c-a3f3-813b86c98d58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:48.011688 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:48.049977 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:48.055398 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18fc89-b753-4304-aa0e-99d1c422c19d None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:48.076237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e864b1e-8254-4c6c-a3f3-813b86c98d58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:48.078112 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e864b1e-8254-4c6c-a3f3-813b86c98d58 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:48.088642 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641639aa-d5fd-4928-abc8-763692757d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:48.089277 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641639aa-d5fd-4928-abc8-763692757d7c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:48.089556 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641639aa-d5fd-4928-abc8-763692757d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:48.089825 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641639aa-d5fd-4928-abc8-763692757d7c None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:48.090456 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641639aa-d5fd-4928-abc8-763692757d7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:48.090755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641639aa-d5fd-4928-abc8-763692757d7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:48.092097 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641639aa-d5fd-4928-abc8-763692757d7c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:48.103562 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 253/512] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:23:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:48.104344 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641639aa-d5fd-4928-abc8-763692757d7c None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:48.181223 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 260/513] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:23:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:23:48.182261 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e864b1e-8254-4c6c-a3f3-813b86c98d58 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:48.203449 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641639aa-d5fd-4928-abc8-763692757d7c None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:48.223007 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:48.236653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1d037-1cf8-4d13-b4b8-e5610166d8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:48.237288 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1d037-1cf8-4d13-b4b8-e5610166d8da None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:48.237578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1d037-1cf8-4d13-b4b8-e5610166d8da None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:48.237900 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1d037-1cf8-4d13-b4b8-e5610166d8da None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:48.238537 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a1d037-1cf8-4d13-b4b8-e5610166d8da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:48.238837 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a1d037-1cf8-4d13-b4b8-e5610166d8da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:48.240280 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a1d037-1cf8-4d13-b4b8-e5610166d8da None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:48.249966 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:48.252456 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 254/514] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:48.253287 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1d037-1cf8-4d13-b4b8-e5610166d8da None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:48.279749 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e864b1e-8254-4c6c-a3f3-813b86c98d58 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:48.315359 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a82ba3-105f-43fc-91e6-c39d6ed8886d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:48.315992 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a82ba3-105f-43fc-91e6-c39d6ed8886d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:48.316268 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a82ba3-105f-43fc-91e6-c39d6ed8886d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:48.316543 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a82ba3-105f-43fc-91e6-c39d6ed8886d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:48.326643 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a82ba3-105f-43fc-91e6-c39d6ed8886d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:48.328268 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a82ba3-105f-43fc-91e6-c39d6ed8886d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:48.343401 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-71a82ba3-105f-43fc-91e6-c39d6ed8886d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:48.343401 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:48.343401 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:48.343401 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:48.343401 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:48.343401 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:48.343401 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:48.343401 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:48.343401 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:48.343401 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:48.343401 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:48.343401 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:48.343401 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:23:48.343401 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:23:48.343401 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:23:48.343401 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:23:48.344322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:48.344322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:48.344322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:48.344322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:48.344322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:48.344322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:48.344322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:48.344322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:48.344322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:48.344322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:48.344322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:48.344322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:48.344322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:48.344322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:48.344322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:48.344322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:48.344322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:23:48.346278 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:23:48.346278 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:48.346278 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:48.346278 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 261/515] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:48.346278 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a82ba3-105f-43fc-91e6-c39d6ed8886d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:48.353930 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1d037-1cf8-4d13-b4b8-e5610166d8da None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:48.386798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e3861-93ee-469f-9c98-c86a2af0a006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:48.387418 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e3861-93ee-469f-9c98-c86a2af0a006 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:48.387692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e3861-93ee-469f-9c98-c86a2af0a006 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:48.387960 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e3861-93ee-469f-9c98-c86a2af0a006 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:48.395270 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e3861-93ee-469f-9c98-c86a2af0a006 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:48.396707 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e3861-93ee-469f-9c98-c86a2af0a006 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:48.410221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-617e3861-93ee-469f-9c98-c86a2af0a006 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 08:23:48.410221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:48.410221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:48.410221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:48.410221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:48.410221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:48.410221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:48.410221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:48.410221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:48.410221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:48.410221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:48.410221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:48.410221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:23:48.410221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:23:48.410221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:23:48.410221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:23:48.411494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:48.411494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:48.411494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:23:48.411494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:23:48.411494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:23:48.411494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:23:48.411494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:48.411494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:48.411494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:48.411494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:48.411494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:48.411494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:48.411494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:48.411494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:48.411494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:48.411494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:48.411494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:48.411494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:48.412406 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:48.412406 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:48.412406 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:23:48.412406 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:23:48.412406 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:23:48.412406 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:23:48.412406 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:23:48.412406 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:23:48.412406 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 08:23:48.412406 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:48.412406 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 255/516] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:48.413286 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e3861-93ee-469f-9c98-c86a2af0a006 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:48.451730 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a82ba3-105f-43fc-91e6-c39d6ed8886d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:48.455444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:48.475454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:48.486561 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b6759-085e-4b30-a3af-3822549cbb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:48.487186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b6759-085e-4b30-a3af-3822549cbb88 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:48.487458 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b6759-085e-4b30-a3af-3822549cbb88 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:48.487724 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b6759-085e-4b30-a3af-3822549cbb88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:48.494389 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b6759-085e-4b30-a3af-3822549cbb88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:48.495925 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b6759-085e-4b30-a3af-3822549cbb88 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:48.507842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-f52b6759-085e-4b30-a3af-3822549cbb88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:23:48.507842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:48.507842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:48.507842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:48.507842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:48.507842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:48.507842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:48.507842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:48.507842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:48.507842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:48.507842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:48.507842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:48.507842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:23:48.507842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:23:48.507842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:23:48.508674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:23:48.508674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:48.508674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:48.508674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:48.508674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:48.508674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:48.508674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:48.508674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:48.508674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:48.508674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:48.508674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:48.508674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:48.508674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:48.508674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:48.508674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:48.508674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:48.508674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:48.509557 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:23:48.509557 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:23:48.509557 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:23:48.509557 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:23:48.509557 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:23:48.509557 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:23:48.509557 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:23:48.509557 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:48.509557 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 262/517] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:23:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:48.510018 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b6759-085e-4b30-a3af-3822549cbb88 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:48.515508 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e3861-93ee-469f-9c98-c86a2af0a006 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:48.548950 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613e210-48f8-4a06-a2e9-fe5526e7f913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:48.549597 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613e210-48f8-4a06-a2e9-fe5526e7f913 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:48.549878 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613e210-48f8-4a06-a2e9-fe5526e7f913 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:48.550151 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613e210-48f8-4a06-a2e9-fe5526e7f913 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:48.561225 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613e210-48f8-4a06-a2e9-fe5526e7f913 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:48.562710 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613e210-48f8-4a06-a2e9-fe5526e7f913 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:48.576689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-1613e210-48f8-4a06-a2e9-fe5526e7f913 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:48.576689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:48.576689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:48.576689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:48.576689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:48.576689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:48.576689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:48.576689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:48.576689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:48.576689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:48.576689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:48.576689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:48.576689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:23:48.576689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:23:48.576689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:23:48.576689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:23:48.578308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:48.578308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:48.578308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:48.578308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:48.578308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:48.578308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:48.578308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:48.578308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:48.578308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:48.578308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:48.578308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:48.578308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:48.578308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:48.578308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:48.578308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:48.578308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:48.578308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:23:48.579733 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:23:48.579733 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:48.579733 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:48.579733 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 256/518] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:48.579733 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613e210-48f8-4a06-a2e9-fe5526e7f913 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:48.610915 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b6759-085e-4b30-a3af-3822549cbb88 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:48.643871 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16159480-4928-419b-976d-4dee4a60da3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:48.644476 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16159480-4928-419b-976d-4dee4a60da3b None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:48.644757 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16159480-4928-419b-976d-4dee4a60da3b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:48.645030 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16159480-4928-419b-976d-4dee4a60da3b None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:48.645691 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16159480-4928-419b-976d-4dee4a60da3b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:48.647137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16159480-4928-419b-976d-4dee4a60da3b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:48.667576 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:48.676515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613e210-48f8-4a06-a2e9-fe5526e7f913 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:48.701049 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 263/519] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 08:23:48 2022] POST /identity/v3/users => generated 247 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:23:48.701949 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16159480-4928-419b-976d-4dee4a60da3b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:48.712816 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f78b8-b988-4904-90b7-af24da3c6862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:48.713469 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f78b8-b988-4904-90b7-af24da3c6862 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:48.713771 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f78b8-b988-4904-90b7-af24da3c6862 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:48.714042 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f78b8-b988-4904-90b7-af24da3c6862 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:48.714698 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039f78b8-b988-4904-90b7-af24da3c6862 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:48.714995 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039f78b8-b988-4904-90b7-af24da3c6862 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:48.716407 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039f78b8-b988-4904-90b7-af24da3c6862 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:48.729806 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 257/520] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:48.730472 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f78b8-b988-4904-90b7-af24da3c6862 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:48.763309 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:48.830806 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16159480-4928-419b-976d-4dee4a60da3b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:48.843312 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f78b8-b988-4904-90b7-af24da3c6862 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:48.874497 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857441c1-13a9-4734-9100-90b49d79ccea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:48.875076 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857441c1-13a9-4734-9100-90b49d79ccea None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:48.875957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857441c1-13a9-4734-9100-90b49d79ccea None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:48.875957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857441c1-13a9-4734-9100-90b49d79ccea None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:48.876343 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857441c1-13a9-4734-9100-90b49d79ccea None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:48.876653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857441c1-13a9-4734-9100-90b49d79ccea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:48.878143 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857441c1-13a9-4734-9100-90b49d79ccea None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:48.881852 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4dfb1b-6533-463f-9429-1010b97f78fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:48.882448 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4dfb1b-6533-463f-9429-1010b97f78fe None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:48.882729 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4dfb1b-6533-463f-9429-1010b97f78fe None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:48.882997 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4dfb1b-6533-463f-9429-1010b97f78fe None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:48.884099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4dfb1b-6533-463f-9429-1010b97f78fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:48.884099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4dfb1b-6533-463f-9429-1010b97f78fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:48.885617 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4dfb1b-6533-463f-9429-1010b97f78fe None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:48.904644 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 258/521] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:23:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:48.906248 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4dfb1b-6533-463f-9429-1010b97f78fe None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:48.919148 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 264/522] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Aug 8 08:23:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:48.920215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857441c1-13a9-4734-9100-90b49d79ccea None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:48.923738 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:49.019164 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4dfb1b-6533-463f-9429-1010b97f78fe None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:49.024412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:49.028309 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857441c1-13a9-4734-9100-90b49d79ccea None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:49.054660 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9426249-ff95-4352-b797-e40cd0e5c0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:49.055213 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9426249-ff95-4352-b797-e40cd0e5c0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:49.055494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9426249-ff95-4352-b797-e40cd0e5c0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:49.055758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9426249-ff95-4352-b797-e40cd0e5c0b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:49.056405 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9426249-ff95-4352-b797-e40cd0e5c0b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:49.056705 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9426249-ff95-4352-b797-e40cd0e5c0b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:49.058114 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9426249-ff95-4352-b797-e40cd0e5c0b8 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:49.064927 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b779ff1-a5c2-401a-ba67-4bfdca0e46db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:49.065630 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b779ff1-a5c2-401a-ba67-4bfdca0e46db None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:49.065914 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b779ff1-a5c2-401a-ba67-4bfdca0e46db None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:49.066189 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b779ff1-a5c2-401a-ba67-4bfdca0e46db None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:49.075261 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 259/523] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:49.075261 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9426249-ff95-4352-b797-e40cd0e5c0b8 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:49.076911 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b779ff1-a5c2-401a-ba67-4bfdca0e46db None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:49.078668 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b779ff1-a5c2-401a-ba67-4bfdca0e46db None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:49.093926 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-6b779ff1-a5c2-401a-ba67-4bfdca0e46db None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 08:23:49.093926 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:49.093926 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:49.093926 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:49.093926 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:49.093926 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:49.093926 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:49.093926 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:49.093926 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:49.093926 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:49.093926 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:49.093926 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:49.093926 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:23:49.093926 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:23:49.093926 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:23:49.093926 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:23:49.095014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:49.095014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:49.095014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:23:49.095014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:23:49.095014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:23:49.095014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:23:49.095014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:49.095014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:49.095014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:49.095014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:49.095014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:49.095014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:49.095014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:49.095014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:49.095014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:49.095014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:49.095014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:49.095014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:49.096469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:49.096469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:49.096469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:23:49.096469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:23:49.096469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:23:49.096469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:23:49.096469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:23:49.096469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:23:49.096469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 08:23:49.096469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:49.096469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 265/524] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 08:23:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:49.100264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b779ff1-a5c2-401a-ba67-4bfdca0e46db None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:49.151570 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:49.182256 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9426249-ff95-4352-b797-e40cd0e5c0b8 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:49.209721 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b779ff1-a5c2-401a-ba67-4bfdca0e46db None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:49.218697 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e04f3-5c3a-4070-8522-299782a2f9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:49.219306 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e04f3-5c3a-4070-8522-299782a2f9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:49.219594 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e04f3-5c3a-4070-8522-299782a2f9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:49.219868 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e04f3-5c3a-4070-8522-299782a2f9b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:49.220607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e04f3-5c3a-4070-8522-299782a2f9b2 None admin] RBAC: Adding query filter params (role.id=402c3e6fe3364285bd72784a297db11e, scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4, user.id=a3317f751ec74bde95ec262394001292) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:49.220916 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e04f3-5c3a-4070-8522-299782a2f9b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:49.222380 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e04f3-5c3a-4070-8522-299782a2f9b2 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:49.228181 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 260/525] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:23:49 2022] GET /identity/v3/role_assignments?user.id=a3317f751ec74bde95ec262394001292&scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4&role.id=402c3e6fe3364285bd72784a297db11e => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:49.228956 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e04f3-5c3a-4070-8522-299782a2f9b2 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:49.257813 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb6bdb-39a0-4133-8dc5-6e40bf5890c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:49.258903 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb6bdb-39a0-4133-8dc5-6e40bf5890c2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:49.259182 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb6bdb-39a0-4133-8dc5-6e40bf5890c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:49.259452 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb6bdb-39a0-4133-8dc5-6e40bf5890c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:49.294857 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:49.353342 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e04f3-5c3a-4070-8522-299782a2f9b2 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:49.359599 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bb6bdb-39a0-4133-8dc5-6e40bf5890c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:49.361429 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bb6bdb-39a0-4133-8dc5-6e40bf5890c2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:49.395907 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b3b82-494c-4528-858f-175999779847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:49.396669 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b3b82-494c-4528-858f-175999779847 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:49.396835 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b3b82-494c-4528-858f-175999779847 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:49.397097 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b3b82-494c-4528-858f-175999779847 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:49.401986 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:49.414109 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563b3b82-494c-4528-858f-175999779847 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:49.415661 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563b3b82-494c-4528-858f-175999779847 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:49.431913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-563b3b82-494c-4528-858f-175999779847 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:23:49.431913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:49.431913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:49.431913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:49.431913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:49.431913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:49.431913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:49.431913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:49.431913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:49.431913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:49.431913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:49.431913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:49.431913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:23:49.431913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:23:49.431913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:23:49.432911 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:23:49.432911 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:49.432911 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:49.432911 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:49.432911 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:49.432911 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:49.432911 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:49.432911 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:49.432911 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:49.432911 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:49.432911 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:49.432911 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:49.432911 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:49.432911 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:49.432911 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:49.432911 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:49.432911 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:49.433809 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:23:49.433809 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:23:49.433809 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:23:49.433809 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:23:49.433809 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:23:49.433809 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:23:49.433809 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:23:49.433809 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:49.438587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 261/526] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:23:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:49.439753 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b3b82-494c-4528-858f-175999779847 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:49.496856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 266/527] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:23:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:23:49.498025 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb6bdb-39a0-4133-8dc5-6e40bf5890c2 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:49.577256 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:49.579080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b3b82-494c-4528-858f-175999779847 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:49.616310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559b339-7f24-4e81-b224-de1fb6d3751a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:49.617308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559b339-7f24-4e81-b224-de1fb6d3751a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:49.617815 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559b339-7f24-4e81-b224-de1fb6d3751a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:49.618440 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559b339-7f24-4e81-b224-de1fb6d3751a None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:49.619471 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e559b339-7f24-4e81-b224-de1fb6d3751a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:49.619975 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e559b339-7f24-4e81-b224-de1fb6d3751a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:49.621654 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e559b339-7f24-4e81-b224-de1fb6d3751a None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:49.621761 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb6bdb-39a0-4133-8dc5-6e40bf5890c2 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:49.646487 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 262/528] 10.222.0.33 () {56 vars in 1139 bytes} [Mon Aug 8 08:23:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:49.647806 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559b339-7f24-4e81-b224-de1fb6d3751a None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:49.658516 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0707e76-51bc-4e2e-a620-4f1af8d34861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:49.659126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0707e76-51bc-4e2e-a620-4f1af8d34861 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:49.659406 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0707e76-51bc-4e2e-a620-4f1af8d34861 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:49.659675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0707e76-51bc-4e2e-a620-4f1af8d34861 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:49.660363 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0707e76-51bc-4e2e-a620-4f1af8d34861 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:49.660666 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0707e76-51bc-4e2e-a620-4f1af8d34861 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:49.662245 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0707e76-51bc-4e2e-a620-4f1af8d34861 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:49.681239 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 267/529] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:23:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:49.681870 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0707e76-51bc-4e2e-a620-4f1af8d34861 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:49.718419 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:49.769131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559b339-7f24-4e81-b224-de1fb6d3751a None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:49.791373 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0707e76-51bc-4e2e-a620-4f1af8d34861 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:49.807100 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0830e53-c83f-4e86-b628-821553e81fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:49.808116 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0830e53-c83f-4e86-b628-821553e81fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:49.808624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0830e53-c83f-4e86-b628-821553e81fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:49.809171 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0830e53-c83f-4e86-b628-821553e81fd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:49.810017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:49.822236 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0830e53-c83f-4e86-b628-821553e81fd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:49.823951 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0830e53-c83f-4e86-b628-821553e81fd3 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:49.829298 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc7c4a-0da9-4f0f-ad0b-e8eac08cc771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:49.829856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc7c4a-0da9-4f0f-ad0b-e8eac08cc771 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:49.830137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc7c4a-0da9-4f0f-ad0b-e8eac08cc771 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:49.830407 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc7c4a-0da9-4f0f-ad0b-e8eac08cc771 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:49.842267 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-a0830e53-c83f-4e86-b628-821553e81fd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:49.842267 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:49.842267 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:49.842267 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:49.842267 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:49.842267 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:49.842267 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:49.842267 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:49.842267 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:49.842267 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:49.842267 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:49.842267 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:49.842267 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:23:49.842267 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:23:49.842267 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:23:49.842267 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:23:49.843295 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:49.843295 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:49.843295 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:49.843295 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:49.843295 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:49.843295 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:49.843295 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:49.843295 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:49.843295 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:49.843295 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:49.843295 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:49.843295 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:49.843295 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:49.843295 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:49.843295 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:49.843295 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:49.843295 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:23:49.844140 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:23:49.844140 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:49.844140 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:49.847041 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 263/530] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:49.848195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0830e53-c83f-4e86-b628-821553e81fd3 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:49.908039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc7c4a-0da9-4f0f-ad0b-e8eac08cc771 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:49.909895 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc7c4a-0da9-4f0f-ad0b-e8eac08cc771 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:49.962188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0830e53-c83f-4e86-b628-821553e81fd3 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:49.970846 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:50.001318 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73277b-3c9b-4e83-b5cb-24e88620bf80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:50.002173 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73277b-3c9b-4e83-b5cb-24e88620bf80 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:50.002579 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73277b-3c9b-4e83-b5cb-24e88620bf80 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:50.002957 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73277b-3c9b-4e83-b5cb-24e88620bf80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:50.003950 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc73277b-3c9b-4e83-b5cb-24e88620bf80 None admin] RBAC: Adding query filter params (role.id=402c3e6fe3364285bd72784a297db11e, scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4, user.id=627171bf2a2e4a21ba6f800879f8c665) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:50.004623 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc73277b-3c9b-4e83-b5cb-24e88620bf80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:50.006523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc73277b-3c9b-4e83-b5cb-24e88620bf80 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:50.012892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 264/531] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:23:49 2022] GET /identity/v3/role_assignments?user.id=627171bf2a2e4a21ba6f800879f8c665&scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4&role.id=402c3e6fe3364285bd72784a297db11e => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:50.013826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73277b-3c9b-4e83-b5cb-24e88620bf80 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:50.022737 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 268/532] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:23:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:23:50.028368 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc7c4a-0da9-4f0f-ad0b-e8eac08cc771 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:50.049424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:50.133288 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73277b-3c9b-4e83-b5cb-24e88620bf80 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:50.143343 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc7c4a-0da9-4f0f-ad0b-e8eac08cc771 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:50.177732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a0843-e968-4ba6-aa8e-bf9ef8bae5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:50.177732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a0843-e968-4ba6-aa8e-bf9ef8bae5fc None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:50.177732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a0843-e968-4ba6-aa8e-bf9ef8bae5fc None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:50.177732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a0843-e968-4ba6-aa8e-bf9ef8bae5fc None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:50.178161 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a0843-e968-4ba6-aa8e-bf9ef8bae5fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:50.178161 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a0843-e968-4ba6-aa8e-bf9ef8bae5fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:50.178161 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a0843-e968-4ba6-aa8e-bf9ef8bae5fc None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:50.191500 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 265/533] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:23:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:50.196876 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a0843-e968-4ba6-aa8e-bf9ef8bae5fc None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:50.214315 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1708a2-51ae-4e5d-b629-3630166c64ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:50.215342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1708a2-51ae-4e5d-b629-3630166c64ca None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:50.215835 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1708a2-51ae-4e5d-b629-3630166c64ca None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:50.216297 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1708a2-51ae-4e5d-b629-3630166c64ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:50.232146 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:50.235611 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1708a2-51ae-4e5d-b629-3630166c64ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:50.237337 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1708a2-51ae-4e5d-b629-3630166c64ca None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:50.254772 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-ef1708a2-51ae-4e5d-b629-3630166c64ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:50.254772 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:50.254772 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:50.254772 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:50.254772 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:50.254772 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:50.254772 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:50.254772 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:50.254772 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:50.254772 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:50.254772 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:50.254772 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:50.254772 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:23:50.254772 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:23:50.254772 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:23:50.254772 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:23:50.255719 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:50.255719 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:50.255719 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:50.255719 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:50.255719 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:50.255719 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:50.255719 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:50.255719 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:50.255719 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:50.255719 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:50.255719 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:50.255719 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:50.255719 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:50.255719 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:50.255719 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:50.255719 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:50.255719 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:23:50.256574 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:23:50.256574 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:50.256574 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:50.258720 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 269/534] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:50.268671 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1708a2-51ae-4e5d-b629-3630166c64ca None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:50.330616 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a0843-e968-4ba6-aa8e-bf9ef8bae5fc None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:50.363978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:50.398789 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1708a2-51ae-4e5d-b629-3630166c64ca None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:50.443948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2d69b-0d8e-49a3-8872-bac24d1450ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:50.444176 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6837cdf-8f78-4b5f-a2eb-e769f9f004a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:50.444610 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2d69b-0d8e-49a3-8872-bac24d1450ad None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:50.444687 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6837cdf-8f78-4b5f-a2eb-e769f9f004a6 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:50.444936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2d69b-0d8e-49a3-8872-bac24d1450ad None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:50.445031 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6837cdf-8f78-4b5f-a2eb-e769f9f004a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:50.445100 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6837cdf-8f78-4b5f-a2eb-e769f9f004a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:50.445206 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2d69b-0d8e-49a3-8872-bac24d1450ad None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:50.445800 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f2d69b-0d8e-49a3-8872-bac24d1450ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:50.446104 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f2d69b-0d8e-49a3-8872-bac24d1450ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:50.447486 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f2d69b-0d8e-49a3-8872-bac24d1450ad None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:50.461430 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 270/535] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:23:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:50.467087 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2d69b-0d8e-49a3-8872-bac24d1450ad None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:50.469934 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6837cdf-8f78-4b5f-a2eb-e769f9f004a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:50.471396 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6837cdf-8f78-4b5f-a2eb-e769f9f004a6 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:50.498593 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-f6837cdf-8f78-4b5f-a2eb-e769f9f004a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:23:50.498593 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:50.498593 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:50.498593 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:50.498593 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:50.498593 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:50.498593 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:50.498593 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:50.498593 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:50.498593 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:50.498593 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:50.498593 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:50.498593 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:23:50.498593 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:23:50.498593 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:23:50.500212 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:23:50.500212 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:50.500212 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:50.500212 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:50.500212 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:50.500212 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:50.500212 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:50.500212 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:50.500212 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:50.500212 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:50.500212 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:50.500212 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:50.500212 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:50.500212 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:50.500212 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:50.500212 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:50.500212 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:50.501176 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:23:50.501176 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:23:50.501176 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:23:50.501176 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:23:50.501176 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:23:50.501176 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:23:50.501176 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:23:50.501176 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:50.505084 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 266/536] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:23:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:23:50.510099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6837cdf-8f78-4b5f-a2eb-e769f9f004a6 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:50.581163 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2d69b-0d8e-49a3-8872-bac24d1450ad None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:50.617950 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:50.632853 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7d7ac-4e03-42de-b3ce-e90b25586cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:50.636641 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7d7ac-4e03-42de-b3ce-e90b25586cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:50.636641 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7d7ac-4e03-42de-b3ce-e90b25586cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:50.636641 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7d7ac-4e03-42de-b3ce-e90b25586cb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:50.653857 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7d7ac-4e03-42de-b3ce-e90b25586cb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:50.655406 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7d7ac-4e03-42de-b3ce-e90b25586cb4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:50.682856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-30e7d7ac-4e03-42de-b3ce-e90b25586cb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:23:50.682856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:50.682856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:50.682856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:50.682856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:50.682856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:50.682856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:50.682856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:50.682856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:50.682856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:50.682856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:50.682856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:50.682856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:23:50.682856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:23:50.682856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:23:50.684879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:23:50.684879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:50.684879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:50.684879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:50.684879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:50.684879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:50.684879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:50.684879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:50.684879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:50.684879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:50.684879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:50.684879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:50.684879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:50.684879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:50.684879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:50.684879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:50.684879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:50.687089 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:23:50.687089 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:23:50.687089 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:23:50.687089 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:23:50.687089 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:23:50.687089 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:23:50.687089 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:23:50.687089 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:50.692785 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 271/537] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:23:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:50.700244 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:50.701954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7d7ac-4e03-42de-b3ce-e90b25586cb4 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:50.719734 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6837cdf-8f78-4b5f-a2eb-e769f9f004a6 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:50.790575 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564fdfb-ad53-4ce9-a1a5-19c057c58b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:50.791237 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564fdfb-ad53-4ce9-a1a5-19c057c58b2e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:50.791528 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564fdfb-ad53-4ce9-a1a5-19c057c58b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:50.791801 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564fdfb-ad53-4ce9-a1a5-19c057c58b2e None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:50.792496 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2564fdfb-ad53-4ce9-a1a5-19c057c58b2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:50.792798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2564fdfb-ad53-4ce9-a1a5-19c057c58b2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:50.798909 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2564fdfb-ad53-4ce9-a1a5-19c057c58b2e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:50.810653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 267/538] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:23:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:50.812886 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5835dd3-d9da-4e94-a20f-29f5e29b4fa7 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:50.813401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5835dd3-d9da-4e94-a20f-29f5e29b4fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:50.813892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5835dd3-d9da-4e94-a20f-29f5e29b4fa7 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:50.816628 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 268/539] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:50.817560 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5835dd3-d9da-4e94-a20f-29f5e29b4fa7 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:50.856874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7d7ac-4e03-42de-b3ce-e90b25586cb4 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:50.900069 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9338c96-7a68-4d42-a845-379804341153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:50.900838 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9338c96-7a68-4d42-a845-379804341153 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:50.901039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9338c96-7a68-4d42-a845-379804341153 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:50.901328 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9338c96-7a68-4d42-a845-379804341153 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:50.902110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9338c96-7a68-4d42-a845-379804341153 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:50.902369 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9338c96-7a68-4d42-a845-379804341153 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:50.903802 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9338c96-7a68-4d42-a845-379804341153 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:50.922918 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 272/540] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:23:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:23:50.922918 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f677492-c71f-4f95-ae66-26774f2b72c7 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:50.922918 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f677492-c71f-4f95-ae66-26774f2b72c7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:50.922918 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f677492-c71f-4f95-ae66-26774f2b72c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:50.939541 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-7f677492-c71f-4f95-ae66-26774f2b72c7 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:50.940930 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7f677492-c71f-4f95-ae66-26774f2b72c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:50.956399 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5835dd3-d9da-4e94-a20f-29f5e29b4fa7 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:51.001502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:51.005365 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ad5c8-018a-4e27-a49f-958f80816287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:51.006450 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ad5c8-018a-4e27-a49f-958f80816287 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:51.006913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ad5c8-018a-4e27-a49f-958f80816287 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:51.007490 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ad5c8-018a-4e27-a49f-958f80816287 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:51.008837 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6ad5c8-018a-4e27-a49f-958f80816287 None admin] RBAC: Adding query filter params (role.id=402c3e6fe3364285bd72784a297db11e, scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4, user.id=3262fa2d9c0d4d8998c9697994b48fef) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:51.009518 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6ad5c8-018a-4e27-a49f-958f80816287 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:51.011620 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6ad5c8-018a-4e27-a49f-958f80816287 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:51.018080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 269/542] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:23:50 2022] GET /identity/v3/role_assignments?user.id=3262fa2d9c0d4d8998c9697994b48fef&scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4&role.id=402c3e6fe3364285bd72784a297db11e => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:51.019913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ad5c8-018a-4e27-a49f-958f80816287 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:51.020182 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 273/542] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:51.038360 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:51.042783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f677492-c71f-4f95-ae66-26774f2b72c7 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:51.195059 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ad5c8-018a-4e27-a49f-958f80816287 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:51.196278 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f677492-c71f-4f95-ae66-26774f2b72c7 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:51.236717 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f092d-78f8-4a6e-bdc7-d64875610da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:51.237988 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f092d-78f8-4a6e-bdc7-d64875610da9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:51.238488 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f092d-78f8-4a6e-bdc7-d64875610da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:51.238957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f092d-78f8-4a6e-bdc7-d64875610da9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:51.253570 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447ac823-28ca-497a-ae5a-5b57447ed036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:51.254300 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ac823-28ca-497a-ae5a-5b57447ed036 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:51.254592 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ac823-28ca-497a-ae5a-5b57447ed036 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:51.254868 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ac823-28ca-497a-ae5a-5b57447ed036 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:51.255619 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447ac823-28ca-497a-ae5a-5b57447ed036 None admin] RBAC: Adding query filter params (role.id=402c3e6fe3364285bd72784a297db11e, scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4, user.id=2c3b8b43026c40209bfc42acd2830610) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:51.255929 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447ac823-28ca-497a-ae5a-5b57447ed036 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:51.257513 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447ac823-28ca-497a-ae5a-5b57447ed036 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:51.264657 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f092d-78f8-4a6e-bdc7-d64875610da9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:51.269825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-378f092d-78f8-4a6e-bdc7-d64875610da9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:23:51.269825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:51.269825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:51.269825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:51.269825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:51.269825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:51.269825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:51.269825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:51.269825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:51.269825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:51.269825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:51.269825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:51.269825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:51.269825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:51.269825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:51.269825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:51.270768 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:51.270768 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:51.270768 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:51.270768 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:51.270768 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:51.270768 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:51.270768 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:51.270768 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:51.270768 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:51.270768 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:51.270768 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:51.270768 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:51.270768 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:51.270768 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:51.270768 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:51.270768 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:51.271564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:51.271564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:51.271564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:51.271564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:51.271564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:51.271564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:51.271564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:51.271564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:51.271564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:51.271564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:51.271564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:23:51.271564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:51.272172 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 270/543] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:23:51 2022] GET /identity/v3/role_assignments?user.id=2c3b8b43026c40209bfc42acd2830610&scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4&role.id=402c3e6fe3364285bd72784a297db11e => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:51.272244 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 274/544] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 08:23:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:23:51.278927 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447ac823-28ca-497a-ae5a-5b57447ed036 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:51.376556 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:51.400841 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447ac823-28ca-497a-ae5a-5b57447ed036 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:51.461788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d417029-b081-4f38-9d9a-724d3e0df6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:51.463233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d417029-b081-4f38-9d9a-724d3e0df6a0 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:51.463725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d417029-b081-4f38-9d9a-724d3e0df6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:51.464225 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d417029-b081-4f38-9d9a-724d3e0df6a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:51.465167 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d417029-b081-4f38-9d9a-724d3e0df6a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:51.465674 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d417029-b081-4f38-9d9a-724d3e0df6a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:51.467279 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d417029-b081-4f38-9d9a-724d3e0df6a0 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:51.481326 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 271/545] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 08:23:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:51.488014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f092d-78f8-4a6e-bdc7-d64875610da9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:51.536585 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:51.614864 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f092d-78f8-4a6e-bdc7-d64875610da9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:51.626879 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814076b6-2e6b-4dcb-ae5c-9113718a40c3 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:51.627165 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814076b6-2e6b-4dcb-ae5c-9113718a40c3 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:51.627442 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814076b6-2e6b-4dcb-ae5c-9113718a40c3 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:51.628899 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 272/546] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:51.634437 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f9473-48c4-4678-a20b-097a52860e1e None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:51.634976 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f9473-48c4-4678-a20b-097a52860e1e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:51.635546 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f9473-48c4-4678-a20b-097a52860e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:51.656925 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-2c9f9473-48c4-4678-a20b-097a52860e1e None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:51.658491 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2c9f9473-48c4-4678-a20b-097a52860e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:51.672251 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f56089-9bb2-4118-b47e-a1984174fe1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:51.672953 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f56089-9bb2-4118-b47e-a1984174fe1e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:51.673257 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f56089-9bb2-4118-b47e-a1984174fe1e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:51.673531 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f56089-9bb2-4118-b47e-a1984174fe1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:51.694981 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f56089-9bb2-4118-b47e-a1984174fe1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:51.696791 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f56089-9bb2-4118-b47e-a1984174fe1e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:51.722290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-a4f56089-9bb2-4118-b47e-a1984174fe1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:51.722290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:51.722290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:51.722290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:51.722290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:51.722290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:51.722290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:51.722290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:51.722290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:51.722290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:51.722290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:51.722290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:51.722290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:23:51.722290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:23:51.722290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:23:51.722290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:23:51.723237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:51.723237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:51.723237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:51.723237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:51.723237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:51.723237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:51.723237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:51.723237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:51.723237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:51.723237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:51.723237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:51.723237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:51.723237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:51.723237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:51.723237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:51.723237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:51.723237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:23:51.724145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:23:51.724145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:51.724145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:51.730325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 275/547] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:51.739212 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f56089-9bb2-4118-b47e-a1984174fe1e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:51.785933 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 273/548] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:23:51.828306 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f9473-48c4-4678-a20b-097a52860e1e None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:51.860738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:51.870313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f56089-9bb2-4118-b47e-a1984174fe1e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:51.907716 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b88b08-ef82-4e05-b4fb-5a80d27805da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:51.908328 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b88b08-ef82-4e05-b4fb-5a80d27805da None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:51.908601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b88b08-ef82-4e05-b4fb-5a80d27805da None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:51.908867 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b88b08-ef82-4e05-b4fb-5a80d27805da None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:51.909541 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b88b08-ef82-4e05-b4fb-5a80d27805da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:51.910065 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b88b08-ef82-4e05-b4fb-5a80d27805da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:51.911420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b88b08-ef82-4e05-b4fb-5a80d27805da None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:51.924605 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 276/549] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:23:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:51.930118 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b88b08-ef82-4e05-b4fb-5a80d27805da None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:51.995968 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:52.038316 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b88b08-ef82-4e05-b4fb-5a80d27805da None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:52.076911 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f9473-48c4-4678-a20b-097a52860e1e None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:52.079213 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c2e31-c954-450c-af8c-8b43b4a645ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:52.079849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c2e31-c954-450c-af8c-8b43b4a645ca None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:52.080125 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c2e31-c954-450c-af8c-8b43b4a645ca None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:52.080394 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c2e31-c954-450c-af8c-8b43b4a645ca None admin] PATH_INFO: `/v3/users/swift` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:52.094452 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c2e31-c954-450c-af8c-8b43b4a645ca None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:52.095910 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c2e31-c954-450c-af8c-8b43b4a645ca None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:52.116285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-784c2e31-c954-450c-af8c-8b43b4a645ca None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 08:23:52.116285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:52.116285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:52.116285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:52.116285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:52.116285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:52.116285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:52.116285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:52.116285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:52.116285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:52.116285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:52.116285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:52.116285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:23:52.116285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:23:52.116285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:23:52.116285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:23:52.118564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:52.118564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:52.118564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:23:52.118564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:23:52.118564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:23:52.118564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:23:52.118564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:52.118564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:52.118564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:52.118564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:52.118564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:52.118564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:52.118564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:52.118564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:52.118564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:52.118564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:52.118564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:52.118564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:52.119789 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:52.119789 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:52.119789 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:23:52.119789 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:23:52.119789 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:23:52.119789 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:23:52.119789 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:23:52.119789 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:23:52.119789 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 08:23:52.119789 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:52.119789 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 277/550] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:23:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:23:52.126331 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c2e31-c954-450c-af8c-8b43b4a645ca None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:52.152366 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f77f45-eda8-4d26-9af5-27a8db6b6b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:52.153013 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f77f45-eda8-4d26-9af5-27a8db6b6b3e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:52.153319 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f77f45-eda8-4d26-9af5-27a8db6b6b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:52.153590 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f77f45-eda8-4d26-9af5-27a8db6b6b3e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:52.156429 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:52.167814 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f77f45-eda8-4d26-9af5-27a8db6b6b3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:52.173662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-d8f77f45-eda8-4d26-9af5-27a8db6b6b3e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:23:52.173662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:52.173662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:52.173662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:52.173662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:52.173662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:52.173662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:52.173662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:52.173662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:52.173662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:52.173662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:52.173662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:52.173662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:52.173662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:52.173662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:52.173662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:52.174788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:52.174788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:52.174788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:52.174788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:52.174788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:52.174788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:52.174788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:52.174788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:52.174788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:52.174788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:52.174788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:52.174788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:52.174788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:52.174788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:52.174788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:52.174788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:52.175605 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:52.175605 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:52.175605 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:52.175605 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:52.175605 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:52.175605 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:52.175605 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:52.175605 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:52.175605 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:52.175605 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:52.175605 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:23:52.175605 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:52.178268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 274/551] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 08:23:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:23:52.183448 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f77f45-eda8-4d26-9af5-27a8db6b6b3e None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:52.333739 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c2e31-c954-450c-af8c-8b43b4a645ca None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:52.361979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f77f45-eda8-4d26-9af5-27a8db6b6b3e None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:52.392765 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9d97c-c702-4fb3-8122-7a35e8e0a112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:52.392765 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9d97c-c702-4fb3-8122-7a35e8e0a112 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:52.392765 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9d97c-c702-4fb3-8122-7a35e8e0a112 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:52.392765 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9d97c-c702-4fb3-8122-7a35e8e0a112 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:52.393206 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a9d97c-c702-4fb3-8122-7a35e8e0a112 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:52.393206 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a9d97c-c702-4fb3-8122-7a35e8e0a112 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:52.393206 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a9d97c-c702-4fb3-8122-7a35e8e0a112 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:52.412821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e42d5-43d6-41e3-9306-9f7582bdb481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:52.418033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 278/552] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Aug 8 08:23:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:52.421687 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e42d5-43d6-41e3-9306-9f7582bdb481 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:52.422080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e42d5-43d6-41e3-9306-9f7582bdb481 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:52.422454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e42d5-43d6-41e3-9306-9f7582bdb481 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:52.423385 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e42d5-43d6-41e3-9306-9f7582bdb481 None admin] RBAC: Adding query filter params (name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:52.424111 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e42d5-43d6-41e3-9306-9f7582bdb481 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:52.424891 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9d97c-c702-4fb3-8122-7a35e8e0a112 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:52.426846 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e42d5-43d6-41e3-9306-9f7582bdb481 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:52.441386 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:52.446267 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 275/553] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 08:23:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:52.451541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e42d5-43d6-41e3-9306-9f7582bdb481 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:52.575333 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9d97c-c702-4fb3-8122-7a35e8e0a112 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:52.576418 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e42d5-43d6-41e3-9306-9f7582bdb481 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:52.577328 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:52.615878 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4e6f90-aba2-4bf2-a101-08dc22a50ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:52.616938 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e6f90-aba2-4bf2-a101-08dc22a50ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:52.617448 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e6f90-aba2-4bf2-a101-08dc22a50ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:52.617927 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e6f90-aba2-4bf2-a101-08dc22a50ff1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:52.621500 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29aa046-3b1c-4157-9396-1d9e77d9ac9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:52.622144 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29aa046-3b1c-4157-9396-1d9e77d9ac9d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:52.622431 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29aa046-3b1c-4157-9396-1d9e77d9ac9d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:52.622711 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29aa046-3b1c-4157-9396-1d9e77d9ac9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:52.642858 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29aa046-3b1c-4157-9396-1d9e77d9ac9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:52.644334 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29aa046-3b1c-4157-9396-1d9e77d9ac9d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:52.663826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-e29aa046-3b1c-4157-9396-1d9e77d9ac9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:52.663826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:52.663826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:52.663826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:52.663826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:52.663826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:52.663826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:52.663826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:52.663826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:52.663826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:52.663826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:52.663826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:52.663826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:23:52.663826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:23:52.663826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:23:52.663826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:23:52.664791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:52.664791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:52.664791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:52.664791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:52.664791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:52.664791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:52.664791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:52.664791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:52.664791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:52.664791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:52.664791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:52.664791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:52.664791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:52.664791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:52.664791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:52.664791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:52.664791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:23:52.665709 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:23:52.665709 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:52.665709 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:52.669214 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 276/554] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:23:52.673112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6c2fa-6b27-4abb-bbe4-7e0b43d3e46a None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:52.673431 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6c2fa-6b27-4abb-bbe4-7e0b43d3e46a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:52.673710 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6c2fa-6b27-4abb-bbe4-7e0b43d3e46a None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:52.677208 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 277/555] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:52.678181 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf6c2fa-6b27-4abb-bbe4-7e0b43d3e46a None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:52.704011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e6f90-aba2-4bf2-a101-08dc22a50ff1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:52.705828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e6f90-aba2-4bf2-a101-08dc22a50ff1 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:52.743305 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:52.782162 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:52.818154 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf6c2fa-6b27-4abb-bbe4-7e0b43d3e46a None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:52.823058 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 279/556] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:23:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:23:52.824583 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc96c9-ee54-4bb3-beff-a6d64471e194 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:52.825074 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc96c9-ee54-4bb3-beff-a6d64471e194 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:52.825657 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc96c9-ee54-4bb3-beff-a6d64471e194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:52.846429 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c6fc96c9-ee54-4bb3-beff-a6d64471e194 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:52.847978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c6fc96c9-ee54-4bb3-beff-a6d64471e194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:52.860077 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224dae0-6906-45fc-86b6-9e0fc96bb341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:52.860966 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224dae0-6906-45fc-86b6-9e0fc96bb341 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:52.861285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224dae0-6906-45fc-86b6-9e0fc96bb341 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:52.861710 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224dae0-6906-45fc-86b6-9e0fc96bb341 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:52.862508 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224dae0-6906-45fc-86b6-9e0fc96bb341 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:52.862814 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224dae0-6906-45fc-86b6-9e0fc96bb341 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:52.864337 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224dae0-6906-45fc-86b6-9e0fc96bb341 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:52.881291 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 278/557] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:23:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:52.882430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224dae0-6906-45fc-86b6-9e0fc96bb341 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:52.942878 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 280/558] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:52.943772 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc96c9-ee54-4bb3-beff-a6d64471e194 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:53.023101 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224dae0-6906-45fc-86b6-9e0fc96bb341 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:53.025992 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:53.074416 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:53.074642 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4e3f60-be6d-45e5-9da3-83319133b72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:53.075474 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e3f60-be6d-45e5-9da3-83319133b72a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:53.075474 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e3f60-be6d-45e5-9da3-83319133b72a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:53.075787 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e3f60-be6d-45e5-9da3-83319133b72a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:53.079605 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc96c9-ee54-4bb3-beff-a6d64471e194 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:53.088432 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4e3f60-be6d-45e5-9da3-83319133b72a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:53.090410 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4e3f60-be6d-45e5-9da3-83319133b72a None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:53.109316 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-fa4e3f60-be6d-45e5-9da3-83319133b72a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:53.109316 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:53.109316 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:53.109316 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:53.109316 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:53.109316 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:53.109316 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:53.109316 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:53.109316 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:53.109316 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:53.109316 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:53.109316 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:53.109316 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:23:53.109316 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:23:53.109316 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:23:53.109316 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:23:53.110362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:53.110362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:53.110362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:53.110362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:53.110362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:53.110362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:53.110362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:53.110362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:53.110362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:53.110362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:53.110362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:53.110362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:53.110362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:53.110362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:53.110362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:53.110362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:53.110362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:23:53.111232 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:23:53.111232 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:53.111232 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:53.115331 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 279/559] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:53.116336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4e3f60-be6d-45e5-9da3-83319133b72a None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:53.129379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ff0e3-9995-4971-be50-f18ec77e8ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:53.130870 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ff0e3-9995-4971-be50-f18ec77e8ffa None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:53.131427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ff0e3-9995-4971-be50-f18ec77e8ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:53.132337 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ff0e3-9995-4971-be50-f18ec77e8ffa None admin] PATH_INFO: `/v3/users/nova` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:53.143729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ff0e3-9995-4971-be50-f18ec77e8ffa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:53.145204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ff0e3-9995-4971-be50-f18ec77e8ffa None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:53.159860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-732ff0e3-9995-4971-be50-f18ec77e8ffa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 08:23:53.159860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:53.159860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:53.159860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:53.159860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:53.159860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:53.159860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:53.159860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:53.159860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:53.159860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:53.159860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:53.159860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:53.159860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:23:53.159860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:23:53.159860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:23:53.159860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:23:53.160849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:53.160849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:53.160849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:23:53.160849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:23:53.160849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:23:53.160849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:23:53.160849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:53.160849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:53.160849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:53.160849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:53.160849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:53.160849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:53.160849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:53.160849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:53.160849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:53.160849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:53.160849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:53.160849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:53.161890 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:53.161890 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:53.161890 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:23:53.161890 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:23:53.161890 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:23:53.161890 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:23:53.161890 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:23:53.161890 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:23:53.161890 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 08:23:53.161890 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:53.165664 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 281/560] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Aug 8 08:23:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:53.166509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ff0e3-9995-4971-be50-f18ec77e8ffa None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:53.244458 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4e3f60-be6d-45e5-9da3-83319133b72a None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:53.279838 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d615b-0ba1-4bc0-973d-8a83611e780a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:53.280976 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d615b-0ba1-4bc0-973d-8a83611e780a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:53.281440 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d615b-0ba1-4bc0-973d-8a83611e780a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:53.281732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d615b-0ba1-4bc0-973d-8a83611e780a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:53.292762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372d615b-0ba1-4bc0-973d-8a83611e780a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:53.298658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-372d615b-0ba1-4bc0-973d-8a83611e780a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:23:53.298658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:53.298658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:53.298658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:53.298658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:53.298658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:53.298658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:53.298658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:53.298658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:53.298658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:53.298658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:53.298658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:53.298658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:53.298658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:53.298658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:53.298658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:53.299588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:53.299588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:53.299588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:53.299588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:53.299588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:53.299588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:53.299588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:53.299588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:53.299588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:53.299588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:53.299588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:53.299588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:53.299588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:53.299588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:53.299588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:53.299588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:53.300447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:53.300447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:53.300447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:53.300447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:53.300447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:53.300447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:53.300447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:53.300447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:53.300447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:53.300447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:53.300447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:23:53.300447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:53.304761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 280/561] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 08:23:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:53.308900 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d615b-0ba1-4bc0-973d-8a83611e780a None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:53.309011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ff0e3-9995-4971-be50-f18ec77e8ffa None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:53.328715 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:53.358529 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9053fa-b13c-4319-b6b7-f01e8865604c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:53.359136 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9053fa-b13c-4319-b6b7-f01e8865604c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:53.359469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9053fa-b13c-4319-b6b7-f01e8865604c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:53.359852 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9053fa-b13c-4319-b6b7-f01e8865604c None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:53.360542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9053fa-b13c-4319-b6b7-f01e8865604c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:53.360912 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9053fa-b13c-4319-b6b7-f01e8865604c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:53.362334 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9053fa-b13c-4319-b6b7-f01e8865604c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:53.376012 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 282/562] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:23:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:53.381348 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9053fa-b13c-4319-b6b7-f01e8865604c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:53.406333 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:53.444121 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d615b-0ba1-4bc0-973d-8a83611e780a None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:53.480428 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e493fc09-9cee-428b-a47f-3325631e6c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:53.481029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493fc09-9cee-428b-a47f-3325631e6c1a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:53.481336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493fc09-9cee-428b-a47f-3325631e6c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:53.481603 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493fc09-9cee-428b-a47f-3325631e6c1a None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:53.482329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e493fc09-9cee-428b-a47f-3325631e6c1a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:53.483046 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e493fc09-9cee-428b-a47f-3325631e6c1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:53.485395 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e493fc09-9cee-428b-a47f-3325631e6c1a None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:53.496022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9053fa-b13c-4319-b6b7-f01e8865604c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:53.503612 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 281/563] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Aug 8 08:23:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:53.504443 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e493fc09-9cee-428b-a47f-3325631e6c1a None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:53.534515 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92136d75-bcaa-41bc-a135-019fb1176280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:53.535434 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92136d75-bcaa-41bc-a135-019fb1176280 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:53.535905 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92136d75-bcaa-41bc-a135-019fb1176280 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:53.536463 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92136d75-bcaa-41bc-a135-019fb1176280 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:53.537129 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92136d75-bcaa-41bc-a135-019fb1176280 None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:53.537464 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92136d75-bcaa-41bc-a135-019fb1176280 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:53.538937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92136d75-bcaa-41bc-a135-019fb1176280 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:53.549574 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 283/564] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 08:23:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:53.551353 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebaf898-841f-4cc1-973f-e37aae259956 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:53.551635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebaf898-841f-4cc1-973f-e37aae259956 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:53.551909 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebaf898-841f-4cc1-973f-e37aae259956 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:53.553287 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 284/565] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:53.553978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebaf898-841f-4cc1-973f-e37aae259956 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:53.584154 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:53.608031 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e493fc09-9cee-428b-a47f-3325631e6c1a None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:53.638302 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:53.644732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808daa1f-ee06-41cf-9cce-9d30e8214947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:53.645381 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808daa1f-ee06-41cf-9cce-9d30e8214947 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:53.645658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808daa1f-ee06-41cf-9cce-9d30e8214947 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:53.645928 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808daa1f-ee06-41cf-9cce-9d30e8214947 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:53.652453 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808daa1f-ee06-41cf-9cce-9d30e8214947 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:53.653946 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808daa1f-ee06-41cf-9cce-9d30e8214947 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:53.657595 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebaf898-841f-4cc1-973f-e37aae259956 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:53.665587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-808daa1f-ee06-41cf-9cce-9d30e8214947 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:23:53.665587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:53.665587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:53.665587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:53.665587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:53.665587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:53.665587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:53.665587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:53.665587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:53.665587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:53.665587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:53.665587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:53.665587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:23:53.665587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:23:53.665587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:23:53.666554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:23:53.666554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:53.666554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:53.666554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:53.666554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:53.666554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:53.666554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:53.666554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:53.666554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:53.666554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:53.666554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:53.666554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:53.666554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:53.666554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:53.666554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:53.666554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:53.666554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:53.667448 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:23:53.667448 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:23:53.667448 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:23:53.667448 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:23:53.667448 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:23:53.667448 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:23:53.667448 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:23:53.667448 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:53.668120 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 282/566] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:23:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:53.669496 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584d8cc-4f4d-4cfd-a287-8ceda7133c92 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:53.669783 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584d8cc-4f4d-4cfd-a287-8ceda7133c92 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:53.670061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584d8cc-4f4d-4cfd-a287-8ceda7133c92 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:53.672199 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 283/567] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:53.672916 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584d8cc-4f4d-4cfd-a287-8ceda7133c92 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:53.694175 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e027c7e-bafa-429d-9875-fd5538097e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:53.694805 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e027c7e-bafa-429d-9875-fd5538097e54 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:53.695082 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e027c7e-bafa-429d-9875-fd5538097e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:53.695355 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e027c7e-bafa-429d-9875-fd5538097e54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:53.705331 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e027c7e-bafa-429d-9875-fd5538097e54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:53.707118 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e027c7e-bafa-429d-9875-fd5538097e54 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:53.723054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-8e027c7e-bafa-429d-9875-fd5538097e54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:53.723054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:53.723054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:53.723054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:53.723054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:53.723054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:53.723054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:53.723054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:53.723054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:53.723054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:53.723054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:53.723054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:53.723054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:23:53.723054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:23:53.723054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:23:53.723054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:23:53.724184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:53.724184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:53.724184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:53.724184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:53.724184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:53.724184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:53.724184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:53.724184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:53.724184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:53.724184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:53.724184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:53.724184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:53.724184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:53.724184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:53.724184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:53.724184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:53.724184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:23:53.725074 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:23:53.725074 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:53.725074 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:53.725074 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 285/568] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:53.725714 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb7545-4cfe-45eb-acc2-b1bdf9c686a4 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:53.725983 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb7545-4cfe-45eb-acc2-b1bdf9c686a4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:53.726252 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb7545-4cfe-45eb-acc2-b1bdf9c686a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:53.744147 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-6fdb7545-4cfe-45eb-acc2-b1bdf9c686a4 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:53.745362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6fdb7545-4cfe-45eb-acc2-b1bdf9c686a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:53.776341 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584d8cc-4f4d-4cfd-a287-8ceda7133c92 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:53.809331 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d1204-24ad-42fc-850c-7979abc53fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:53.809952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d1204-24ad-42fc-850c-7979abc53fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:53.810233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d1204-24ad-42fc-850c-7979abc53fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:53.810506 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d1204-24ad-42fc-850c-7979abc53fc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:53.811626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 286/569] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:53.812391 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:53.812497 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb7545-4cfe-45eb-acc2-b1bdf9c686a4 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:53.830462 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d1204-24ad-42fc-850c-7979abc53fc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:53.832223 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d1204-24ad-42fc-850c-7979abc53fc0 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:53.847951 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-4d3d1204-24ad-42fc-850c-7979abc53fc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:53.847951 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:53.847951 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:53.847951 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:53.847951 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:53.847951 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:53.847951 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:53.847951 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:53.847951 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:53.847951 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:53.847951 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:53.847951 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:53.847951 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:23:53.847951 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:23:53.847951 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:23:53.847951 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:23:53.849105 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:53.849105 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:53.849105 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:53.849105 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:53.849105 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:53.849105 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:53.849105 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:53.849105 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:53.849105 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:53.849105 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:53.849105 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:53.849105 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:53.849105 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:53.849105 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:53.849105 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:53.849105 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:53.849105 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:23:53.850093 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:23:53.850093 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:53.850093 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:53.850093 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 284/570] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:53.850593 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427889c-6365-498c-8b33-054e726abc2a None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:53.850878 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427889c-6365-498c-8b33-054e726abc2a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:53.851153 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427889c-6365-498c-8b33-054e726abc2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:53.869810 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-0427889c-6365-498c-8b33-054e726abc2a None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:53.870991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0427889c-6365-498c-8b33-054e726abc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:53.875571 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:53.913896 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb7545-4cfe-45eb-acc2-b1bdf9c686a4 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:53.937851 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 285/571] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:53.938697 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427889c-6365-498c-8b33-054e726abc2a None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:53.953311 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be33d4c-95af-4191-89f9-a9e3cecedb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:53.953922 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be33d4c-95af-4191-89f9-a9e3cecedb2f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:53.954198 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be33d4c-95af-4191-89f9-a9e3cecedb2f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:53.954465 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be33d4c-95af-4191-89f9-a9e3cecedb2f None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:53.955151 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be33d4c-95af-4191-89f9-a9e3cecedb2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:53.955446 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be33d4c-95af-4191-89f9-a9e3cecedb2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:53.957136 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be33d4c-95af-4191-89f9-a9e3cecedb2f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:53.969882 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 287/572] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:23:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:53.970680 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be33d4c-95af-4191-89f9-a9e3cecedb2f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:54.054475 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427889c-6365-498c-8b33-054e726abc2a None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:54.070976 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:54.073224 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be33d4c-95af-4191-89f9-a9e3cecedb2f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:54.088620 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2d103-1784-432d-88b8-fb20871f26fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:54.089257 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2d103-1784-432d-88b8-fb20871f26fc None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:54.092078 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2d103-1784-432d-88b8-fb20871f26fc None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:54.092078 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2d103-1784-432d-88b8-fb20871f26fc None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:54.092078 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2d103-1784-432d-88b8-fb20871f26fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:54.092078 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2d103-1784-432d-88b8-fb20871f26fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:54.092078 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2d103-1784-432d-88b8-fb20871f26fc None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:54.098235 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:54.103956 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 286/573] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:23:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:54.104676 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2d103-1784-432d-88b8-fb20871f26fc None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:54.112739 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6322dc-b930-4fd9-8620-7aeca789c8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:54.113733 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6322dc-b930-4fd9-8620-7aeca789c8d7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:54.113733 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6322dc-b930-4fd9-8620-7aeca789c8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:54.113959 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6322dc-b930-4fd9-8620-7aeca789c8d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:54.124729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6322dc-b930-4fd9-8620-7aeca789c8d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:54.129817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-fb6322dc-b930-4fd9-8620-7aeca789c8d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:23:54.129817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:54.129817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:54.129817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:54.129817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:54.129817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:54.129817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:54.129817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:54.129817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:54.129817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:54.129817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:54.129817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:54.129817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:54.129817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:54.129817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:54.129817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:54.130941 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:54.130941 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:54.130941 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:54.130941 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:54.130941 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:54.130941 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:54.130941 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:54.130941 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:54.130941 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:54.130941 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:54.130941 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:54.130941 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:54.130941 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:54.130941 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:54.130941 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:54.130941 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:54.131919 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:54.131919 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:54.131919 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:54.131919 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:54.131919 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:54.131919 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:54.131919 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:54.131919 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:54.131919 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:54.131919 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:54.131919 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:23:54.131919 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:54.131919 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 288/574] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 08:23:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:54.131919 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6322dc-b930-4fd9-8620-7aeca789c8d7 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:54.207002 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2d103-1784-432d-88b8-fb20871f26fc None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:54.232595 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6322dc-b930-4fd9-8620-7aeca789c8d7 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:54.240977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba04f4-2a84-4934-8b8e-50438d438cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:54.241611 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba04f4-2a84-4934-8b8e-50438d438cbe None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:54.241890 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba04f4-2a84-4934-8b8e-50438d438cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:54.242164 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba04f4-2a84-4934-8b8e-50438d438cbe None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:54.242828 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba04f4-2a84-4934-8b8e-50438d438cbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:54.243128 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba04f4-2a84-4934-8b8e-50438d438cbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:54.244505 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba04f4-2a84-4934-8b8e-50438d438cbe None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:54.256995 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 287/575] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:54.257817 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba04f4-2a84-4934-8b8e-50438d438cbe None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:54.270146 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e07e18-a8f5-489e-bae3-aab8b5edf410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:54.270772 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e07e18-a8f5-489e-bae3-aab8b5edf410 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:54.271051 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e07e18-a8f5-489e-bae3-aab8b5edf410 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:54.271324 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e07e18-a8f5-489e-bae3-aab8b5edf410 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:54.281861 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e07e18-a8f5-489e-bae3-aab8b5edf410 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:54.286977 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-74e07e18-a8f5-489e-bae3-aab8b5edf410 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:23:54.286977 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:54.286977 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:54.286977 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:54.286977 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:54.286977 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:54.286977 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:54.286977 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:54.286977 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:54.286977 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:54.286977 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:54.286977 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:54.286977 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:54.286977 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:54.286977 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:54.286977 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:54.287939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:54.287939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:54.287939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:54.287939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:54.287939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:54.287939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:54.287939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:54.287939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:54.287939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:54.287939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:54.287939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:54.287939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:54.287939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:54.287939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:54.287939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:54.287939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:54.288967 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:54.288967 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:54.288967 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:54.288967 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:54.288967 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:54.288967 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:54.288967 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:54.288967 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:54.288967 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:54.288967 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:54.288967 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:23:54.288967 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:54.288967 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 289/576] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 08:23:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:54.288967 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e07e18-a8f5-489e-bae3-aab8b5edf410 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:54.295542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:54.336748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:54.360486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba04f4-2a84-4934-8b8e-50438d438cbe None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:54.390639 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e07e18-a8f5-489e-bae3-aab8b5edf410 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:54.393407 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4683bf-7826-4642-92b1-5b11c7a79a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:54.394004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4683bf-7826-4642-92b1-5b11c7a79a42 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:54.394281 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4683bf-7826-4642-92b1-5b11c7a79a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:54.394555 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4683bf-7826-4642-92b1-5b11c7a79a42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:54.395267 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4683bf-7826-4642-92b1-5b11c7a79a42 None admin] RBAC: Adding query filter params (role.id=402c3e6fe3364285bd72784a297db11e, scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4, user.id=e69c0e02ad944f2a85fc5318018e5cb0) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:54.395570 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4683bf-7826-4642-92b1-5b11c7a79a42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:54.396978 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4683bf-7826-4642-92b1-5b11c7a79a42 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:54.402310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 288/577] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:23:54 2022] GET /identity/v3/role_assignments?user.id=e69c0e02ad944f2a85fc5318018e5cb0&scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4&role.id=402c3e6fe3364285bd72784a297db11e => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:54.403093 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4683bf-7826-4642-92b1-5b11c7a79a42 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:54.428585 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f01b4f-073b-4461-90b4-56a0085b5261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:54.429273 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f01b4f-073b-4461-90b4-56a0085b5261 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:54.429549 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f01b4f-073b-4461-90b4-56a0085b5261 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:54.429815 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f01b4f-073b-4461-90b4-56a0085b5261 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:54.438038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f01b4f-073b-4461-90b4-56a0085b5261 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:54.439526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f01b4f-073b-4461-90b4-56a0085b5261 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:54.451449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-63f01b4f-073b-4461-90b4-56a0085b5261 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:23:54.451449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:54.451449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:54.451449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:54.451449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:54.451449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:54.451449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:54.451449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:54.451449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:54.451449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:54.451449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:54.451449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:54.451449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:23:54.451449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:23:54.451449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:23:54.452477 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:23:54.452477 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:54.452477 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:54.452477 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:54.452477 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:54.452477 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:54.452477 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:54.452477 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:54.452477 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:54.452477 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:54.452477 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:54.452477 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:54.452477 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:54.452477 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:54.452477 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:54.452477 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:54.452477 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:54.454499 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:23:54.454499 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:23:54.454499 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:23:54.454499 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:23:54.454499 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:23:54.454499 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:23:54.454499 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:23:54.454499 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:54.457590 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 290/578] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:23:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:54.459578 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f01b4f-073b-4461-90b4-56a0085b5261 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:54.519853 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:54.523395 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4683bf-7826-4642-92b1-5b11c7a79a42 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:54.562918 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a9c463-3b11-4bde-a706-58f4602279a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:54.563610 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9c463-3b11-4bde-a706-58f4602279a6 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:54.563886 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9c463-3b11-4bde-a706-58f4602279a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:54.565418 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9c463-3b11-4bde-a706-58f4602279a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:54.565418 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a9c463-3b11-4bde-a706-58f4602279a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:54.565418 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a9c463-3b11-4bde-a706-58f4602279a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:54.566453 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a9c463-3b11-4bde-a706-58f4602279a6 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:54.571739 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f01b4f-073b-4461-90b4-56a0085b5261 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:54.574068 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:54.589479 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 289/579] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 08:23:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:54.593595 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a9c463-3b11-4bde-a706-58f4602279a6 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:54.610518 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f84ea-3a31-45ea-b04b-b31cceb90ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:54.611154 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f84ea-3a31-45ea-b04b-b31cceb90ade None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:54.611437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f84ea-3a31-45ea-b04b-b31cceb90ade None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:54.611715 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f84ea-3a31-45ea-b04b-b31cceb90ade None admin] PATH_INFO: `/v3/users/glance` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:54.619404 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f84ea-3a31-45ea-b04b-b31cceb90ade None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:54.620866 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f84ea-3a31-45ea-b04b-b31cceb90ade None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:54.632102 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-b22f84ea-3a31-45ea-b04b-b31cceb90ade None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 08:23:54.632102 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:54.632102 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:54.632102 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:54.632102 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:54.632102 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:54.632102 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:54.632102 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:54.632102 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:54.632102 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:54.632102 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:54.632102 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:54.632102 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:23:54.632102 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:23:54.632102 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:23:54.632102 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:23:54.633043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:54.633043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:54.633043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:23:54.633043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:23:54.633043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:23:54.633043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:23:54.633043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:54.633043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:54.633043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:54.633043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:54.633043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:54.633043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:54.633043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:54.633043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:54.633043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:54.633043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:54.633043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:54.633043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:54.634024 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:54.634024 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:54.634024 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:23:54.634024 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:23:54.634024 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:23:54.634024 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:23:54.634024 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:23:54.634024 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:23:54.634024 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 08:23:54.634024 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:54.634024 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 291/580] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:54.634024 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f84ea-3a31-45ea-b04b-b31cceb90ade None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:54.703666 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a9c463-3b11-4bde-a706-58f4602279a6 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:54.740164 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26098856-01fc-4f15-ba06-78169f1503ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:54.741004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26098856-01fc-4f15-ba06-78169f1503ca None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:54.741352 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26098856-01fc-4f15-ba06-78169f1503ca None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:54.741579 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26098856-01fc-4f15-ba06-78169f1503ca None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:54.742241 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26098856-01fc-4f15-ba06-78169f1503ca None admin] RBAC: Adding query filter params (name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:54.742536 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26098856-01fc-4f15-ba06-78169f1503ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:54.743877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26098856-01fc-4f15-ba06-78169f1503ca None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:54.744224 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f84ea-3a31-45ea-b04b-b31cceb90ade None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:54.752007 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 290/581] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 08:23:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:54.752803 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26098856-01fc-4f15-ba06-78169f1503ca None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:54.778671 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:54.787195 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be1e45-9462-4665-aa39-0146ae905b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:54.788060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be1e45-9462-4665-aa39-0146ae905b85 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:54.789181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be1e45-9462-4665-aa39-0146ae905b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:54.789181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be1e45-9462-4665-aa39-0146ae905b85 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:54.789731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be1e45-9462-4665-aa39-0146ae905b85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:54.790175 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be1e45-9462-4665-aa39-0146ae905b85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:54.791807 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be1e45-9462-4665-aa39-0146ae905b85 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:54.807040 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 292/582] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:23:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:54.808105 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be1e45-9462-4665-aa39-0146ae905b85 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:54.838262 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:54.865486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26098856-01fc-4f15-ba06-78169f1503ca None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:54.901824 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c744f-5665-408f-8c7f-e025234fb3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:54.902455 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c744f-5665-408f-8c7f-e025234fb3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:54.902737 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c744f-5665-408f-8c7f-e025234fb3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:54.903014 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c744f-5665-408f-8c7f-e025234fb3c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:54.914892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c744f-5665-408f-8c7f-e025234fb3c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:54.916347 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c744f-5665-408f-8c7f-e025234fb3c9 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:54.927091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be1e45-9462-4665-aa39-0146ae905b85 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:54.932028 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-b22c744f-5665-408f-8c7f-e025234fb3c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:54.932028 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:54.932028 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:54.932028 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:54.932028 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:54.932028 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:54.932028 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:54.932028 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:54.932028 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:54.932028 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:54.932028 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:54.932028 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:54.932028 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:23:54.932028 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:23:54.932028 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:23:54.932028 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:23:54.932967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:54.932967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:54.932967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:54.932967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:54.932967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:54.932967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:54.932967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:54.932967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:54.932967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:54.932967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:54.932967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:54.932967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:54.932967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:54.932967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:54.932967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:54.932967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:54.932967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:23:54.934189 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:23:54.934189 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:54.934189 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:54.937479 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 291/583] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:54.938627 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c744f-5665-408f-8c7f-e025234fb3c9 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:54.964459 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0852298-c4fe-40bc-a5c7-ac48e1507622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:54.965090 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0852298-c4fe-40bc-a5c7-ac48e1507622 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:54.965370 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0852298-c4fe-40bc-a5c7-ac48e1507622 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:54.965642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0852298-c4fe-40bc-a5c7-ac48e1507622 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:54.966363 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0852298-c4fe-40bc-a5c7-ac48e1507622 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:54.966666 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0852298-c4fe-40bc-a5c7-ac48e1507622 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:54.968886 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0852298-c4fe-40bc-a5c7-ac48e1507622 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:54.991251 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 293/584] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Aug 8 08:23:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:54.991251 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0852298-c4fe-40bc-a5c7-ac48e1507622 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:55.026514 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:55.051023 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c744f-5665-408f-8c7f-e025234fb3c9 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:55.082306 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:55.087368 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be664bcb-d155-469f-9eb9-a40a751a69c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:55.088109 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be664bcb-d155-469f-9eb9-a40a751a69c9 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:55.088503 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be664bcb-d155-469f-9eb9-a40a751a69c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:55.088881 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be664bcb-d155-469f-9eb9-a40a751a69c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:55.100990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be664bcb-d155-469f-9eb9-a40a751a69c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:55.101139 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0852298-c4fe-40bc-a5c7-ac48e1507622 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:55.102684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be664bcb-d155-469f-9eb9-a40a751a69c9 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:55.117470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-be664bcb-d155-469f-9eb9-a40a751a69c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:55.117470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:55.117470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:55.117470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:55.117470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:55.117470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:55.117470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:55.117470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:55.117470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:55.117470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:55.117470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:55.117470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:55.117470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:23:55.117470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:23:55.117470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:23:55.117470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:23:55.118613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:55.118613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:55.118613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:55.118613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:55.118613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:55.118613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:55.118613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:55.118613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:55.118613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:55.118613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:55.118613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:55.118613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:55.118613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:55.118613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:55.118613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:55.118613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:55.118613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:23:55.119664 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:23:55.119664 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:55.119664 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:55.123492 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 292/585] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:55.125024 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be664bcb-d155-469f-9eb9-a40a751a69c9 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:55.143573 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28029360-cbc6-45d0-b2bf-e8c42cfcab20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:55.144545 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28029360-cbc6-45d0-b2bf-e8c42cfcab20 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:55.145212 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28029360-cbc6-45d0-b2bf-e8c42cfcab20 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:55.145638 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28029360-cbc6-45d0-b2bf-e8c42cfcab20 None admin] PATH_INFO: `/v3/projects/71e5e82c9f454cdd91dfeac5a57968c4/users/a3317f751ec74bde95ec262394001292/roles/402c3e6fe3364285bd72784a297db11e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:55.163748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28029360-cbc6-45d0-b2bf-e8c42cfcab20 None admin] RBAC: Authorizing `identity:create_grant(project_id=71e5e82c9f454cdd91dfeac5a57968c4, user_id=a3317f751ec74bde95ec262394001292, role_id=402c3e6fe3364285bd72784a297db11e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:55.165332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28029360-cbc6-45d0-b2bf-e8c42cfcab20 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:55.215345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 294/586] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 08:23:54 2022] PUT /identity/v3/projects/71e5e82c9f454cdd91dfeac5a57968c4/users/a3317f751ec74bde95ec262394001292/roles/402c3e6fe3364285bd72784a297db11e => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:23:55.216358 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28029360-cbc6-45d0-b2bf-e8c42cfcab20 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:55.250215 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be664bcb-d155-469f-9eb9-a40a751a69c9 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:55.282028 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:55.296049 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481f707-b0f6-41c2-ac5f-21b15d01b75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:55.296664 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481f707-b0f6-41c2-ac5f-21b15d01b75b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:55.297015 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481f707-b0f6-41c2-ac5f-21b15d01b75b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:55.297310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481f707-b0f6-41c2-ac5f-21b15d01b75b None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:55.297964 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5481f707-b0f6-41c2-ac5f-21b15d01b75b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:55.298258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5481f707-b0f6-41c2-ac5f-21b15d01b75b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:55.299643 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5481f707-b0f6-41c2-ac5f-21b15d01b75b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:55.322823 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 293/587] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:23:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:55.324350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481f707-b0f6-41c2-ac5f-21b15d01b75b None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:55.365010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28029360-cbc6-45d0-b2bf-e8c42cfcab20 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:55.401255 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:55.406690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3a679-45b5-4329-bf0e-da9dd2964d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:55.407331 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3a679-45b5-4329-bf0e-da9dd2964d96 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:55.407606 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3a679-45b5-4329-bf0e-da9dd2964d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:55.407879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3a679-45b5-4329-bf0e-da9dd2964d96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:55.425094 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d3a679-45b5-4329-bf0e-da9dd2964d96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:55.426622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d3a679-45b5-4329-bf0e-da9dd2964d96 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:55.443687 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-50d3a679-45b5-4329-bf0e-da9dd2964d96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:55.443687 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:55.443687 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:55.443687 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:55.443687 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:55.443687 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:55.443687 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:55.443687 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:55.443687 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:55.443687 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:55.443687 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:55.443687 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:55.443687 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:23:55.443687 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:23:55.443687 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:23:55.443687 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:23:55.445017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:55.445017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:55.445017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:55.445017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:55.445017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:55.445017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:55.445017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:55.445017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:55.445017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:55.445017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:55.445017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:55.445017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:55.445017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:55.445017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:55.445017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:55.445017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:55.445017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:23:55.446004 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:23:55.446004 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:55.446004 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:55.447202 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481f707-b0f6-41c2-ac5f-21b15d01b75b None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:55.449224 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 295/588] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:55.451252 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3a679-45b5-4329-bf0e-da9dd2964d96 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:55.487229 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff699f9-99d7-40b4-b19f-a9e68c502cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:55.487649 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff699f9-99d7-40b4-b19f-a9e68c502cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:55.487937 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff699f9-99d7-40b4-b19f-a9e68c502cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:55.488208 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff699f9-99d7-40b4-b19f-a9e68c502cb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:55.488870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff699f9-99d7-40b4-b19f-a9e68c502cb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:55.489173 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff699f9-99d7-40b4-b19f-a9e68c502cb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:55.490620 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff699f9-99d7-40b4-b19f-a9e68c502cb3 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:55.504167 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 294/589] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:23:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:55.509566 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff699f9-99d7-40b4-b19f-a9e68c502cb3 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:55.579727 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:55.580906 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3a679-45b5-4329-bf0e-da9dd2964d96 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:55.626565 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c7330-62d2-4dbb-9fb4-12447ee4a14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:55.627591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c7330-62d2-4dbb-9fb4-12447ee4a14b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:55.628569 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c7330-62d2-4dbb-9fb4-12447ee4a14b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:55.629038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c7330-62d2-4dbb-9fb4-12447ee4a14b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:55.637688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c7330-62d2-4dbb-9fb4-12447ee4a14b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:55.639400 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c7330-62d2-4dbb-9fb4-12447ee4a14b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:55.640569 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff699f9-99d7-40b4-b19f-a9e68c502cb3 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:55.652186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-6d1c7330-62d2-4dbb-9fb4-12447ee4a14b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 08:23:55.652186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:55.652186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:55.652186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:55.652186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:55.652186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:55.652186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:55.652186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:55.652186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:55.652186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:55.652186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:55.652186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:55.652186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:23:55.652186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:23:55.652186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:23:55.652186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:23:55.653138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:55.653138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:55.653138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:23:55.653138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:23:55.653138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:23:55.653138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:23:55.653138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:55.653138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:55.653138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:55.653138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:55.653138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:55.653138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:55.653138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:55.653138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:55.653138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:55.653138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:55.653138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:55.653138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:55.654449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:55.654449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:55.654449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:23:55.654449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:23:55.654449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:23:55.654449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:23:55.654449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:23:55.654449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:23:55.654449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 08:23:55.654449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:55.657647 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 296/590] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:23:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:55.658675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c7330-62d2-4dbb-9fb4-12447ee4a14b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:55.680902 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4b063-2dca-4db2-94f6-6d3d034d3d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:55.681602 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4b063-2dca-4db2-94f6-6d3d034d3d3c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:55.681888 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4b063-2dca-4db2-94f6-6d3d034d3d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:55.682164 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4b063-2dca-4db2-94f6-6d3d034d3d3c None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:55.682981 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d4b063-2dca-4db2-94f6-6d3d034d3d3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:55.683291 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d4b063-2dca-4db2-94f6-6d3d034d3d3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:55.684768 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d4b063-2dca-4db2-94f6-6d3d034d3d3c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:55.698765 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 295/591] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:23:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:55.699290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4b063-2dca-4db2-94f6-6d3d034d3d3c None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:55.710819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:55.769802 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c7330-62d2-4dbb-9fb4-12447ee4a14b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:55.805597 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e477e63-bb2e-4fcf-b641-db059cce9d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:55.806604 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e477e63-bb2e-4fcf-b641-db059cce9d54 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:55.807083 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e477e63-bb2e-4fcf-b641-db059cce9d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:55.807547 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e477e63-bb2e-4fcf-b641-db059cce9d54 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:55.816031 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4b063-2dca-4db2-94f6-6d3d034d3d3c None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:55.816398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e477e63-bb2e-4fcf-b641-db059cce9d54 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:55.818281 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e477e63-bb2e-4fcf-b641-db059cce9d54 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:55.826542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:55.831174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-2e477e63-bb2e-4fcf-b641-db059cce9d54 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 08:23:55.831174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:55.831174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:55.831174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:55.831174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:55.831174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:55.831174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:55.831174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:55.831174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:55.831174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:55.831174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:55.831174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:55.831174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:23:55.831174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:23:55.831174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:23:55.831174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:23:55.832350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:55.832350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:55.832350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:23:55.832350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:23:55.832350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:23:55.832350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:23:55.832350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:55.832350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:55.832350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:55.832350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:55.832350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:55.832350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:55.832350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:55.832350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:55.832350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:55.832350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:55.832350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:55.832350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:55.833291 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:55.833291 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:55.833291 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:23:55.833291 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:23:55.833291 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:23:55.833291 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:23:55.833291 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:23:55.833291 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:23:55.833291 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 08:23:55.833291 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:55.835728 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 297/592] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 08:23:55 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:55.836702 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e477e63-bb2e-4fcf-b641-db059cce9d54 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:55.857431 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f0901-b350-4743-b9c4-d405afc0146b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:55.859257 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f0901-b350-4743-b9c4-d405afc0146b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:55.861162 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f0901-b350-4743-b9c4-d405afc0146b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:55.861468 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f0901-b350-4743-b9c4-d405afc0146b None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:55.862213 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119f0901-b350-4743-b9c4-d405afc0146b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:55.862517 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119f0901-b350-4743-b9c4-d405afc0146b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:55.864791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119f0901-b350-4743-b9c4-d405afc0146b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:55.885984 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 296/593] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Aug 8 08:23:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:55.887086 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f0901-b350-4743-b9c4-d405afc0146b None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:55.962337 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e477e63-bb2e-4fcf-b641-db059cce9d54 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:55.964876 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:55.997124 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f0901-b350-4743-b9c4-d405afc0146b None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:56.010310 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830ff86-d653-44b0-a5a9-c3bd44036f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:56.010945 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830ff86-d653-44b0-a5a9-c3bd44036f90 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:56.011221 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830ff86-d653-44b0-a5a9-c3bd44036f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:56.011489 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830ff86-d653-44b0-a5a9-c3bd44036f90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:56.019436 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1830ff86-d653-44b0-a5a9-c3bd44036f90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:56.021008 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1830ff86-d653-44b0-a5a9-c3bd44036f90 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:56.032568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-1830ff86-d653-44b0-a5a9-c3bd44036f90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:23:56.032568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:56.032568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:56.032568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:56.032568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:56.032568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:56.032568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:56.032568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:56.032568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:56.032568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:56.032568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:56.032568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:56.032568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:23:56.032568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:23:56.032568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:23:56.033524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:23:56.033524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:56.033524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:56.033524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:56.033524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:56.033524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:56.033524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:56.033524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:56.033524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:56.033524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:56.033524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:56.033524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:56.033524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:56.033524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:56.033524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:56.033524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:56.033524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:56.034427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:23:56.034427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:23:56.034427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:23:56.034427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:23:56.034427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:23:56.034427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:23:56.034427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:23:56.034427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:56.038387 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 298/594] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:23:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:56.039247 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830ff86-d653-44b0-a5a9-c3bd44036f90 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:56.046741 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5447034f-5ac5-4ea5-b300-19407caf00d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:56.047643 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447034f-5ac5-4ea5-b300-19407caf00d3 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:56.048110 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447034f-5ac5-4ea5-b300-19407caf00d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:56.048636 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447034f-5ac5-4ea5-b300-19407caf00d3 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:56.049603 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5447034f-5ac5-4ea5-b300-19407caf00d3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:56.050624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5447034f-5ac5-4ea5-b300-19407caf00d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:56.052686 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5447034f-5ac5-4ea5-b300-19407caf00d3 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:56.074552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 297/595] 10.222.0.33 () {56 vars in 1139 bytes} [Mon Aug 8 08:23:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:56.075537 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5447034f-5ac5-4ea5-b300-19407caf00d3 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:56.105540 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:56.160688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830ff86-d653-44b0-a5a9-c3bd44036f90 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:56.195379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a731aa6-2d4a-4cfe-9b29-fbf539c64d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:56.196005 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a731aa6-2d4a-4cfe-9b29-fbf539c64d62 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:56.196279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a731aa6-2d4a-4cfe-9b29-fbf539c64d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:56.196555 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a731aa6-2d4a-4cfe-9b29-fbf539c64d62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:56.208655 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a731aa6-2d4a-4cfe-9b29-fbf539c64d62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:56.210137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a731aa6-2d4a-4cfe-9b29-fbf539c64d62 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:56.210943 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5447034f-5ac5-4ea5-b300-19407caf00d3 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:56.225386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-0a731aa6-2d4a-4cfe-9b29-fbf539c64d62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:56.225386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:56.225386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:56.225386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:56.225386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:56.225386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:56.225386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:56.225386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:56.225386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:56.225386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:56.225386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:56.225386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:56.225386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:23:56.225386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:23:56.225386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:23:56.225386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:23:56.226351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:56.226351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:56.226351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:56.226351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:56.226351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:56.226351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:56.226351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:56.226351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:56.226351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:56.226351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:56.226351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:56.226351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:56.226351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:56.226351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:56.226351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:56.226351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:56.226351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:23:56.227256 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:23:56.227256 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:56.227256 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:56.227256 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 299/596] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:56.227918 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a731aa6-2d4a-4cfe-9b29-fbf539c64d62 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:56.248135 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:56.266363 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329def8f-b16b-405b-af23-2352498f1e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:56.266991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329def8f-b16b-405b-af23-2352498f1e66 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:56.267278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329def8f-b16b-405b-af23-2352498f1e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:56.267556 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329def8f-b16b-405b-af23-2352498f1e66 None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:56.268256 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329def8f-b16b-405b-af23-2352498f1e66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:56.268561 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329def8f-b16b-405b-af23-2352498f1e66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:56.270008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329def8f-b16b-405b-af23-2352498f1e66 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:56.286378 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 298/597] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:23:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:56.287159 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329def8f-b16b-405b-af23-2352498f1e66 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:56.348054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a731aa6-2d4a-4cfe-9b29-fbf539c64d62 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:56.392664 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b158be-1b66-41c8-9e9d-29dc37ff149c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:56.393827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b158be-1b66-41c8-9e9d-29dc37ff149c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:56.394321 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b158be-1b66-41c8-9e9d-29dc37ff149c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:56.394785 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b158be-1b66-41c8-9e9d-29dc37ff149c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:56.400606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:56.406163 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b158be-1b66-41c8-9e9d-29dc37ff149c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:56.409589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b158be-1b66-41c8-9e9d-29dc37ff149c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:56.428043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-54b158be-1b66-41c8-9e9d-29dc37ff149c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:56.428043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:56.428043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:56.428043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:56.428043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:56.428043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:56.428043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:56.428043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:56.428043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:56.428043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:56.428043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:56.428043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:56.428043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:23:56.428043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:23:56.428043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:23:56.428043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:23:56.429405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:56.429405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:56.429405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:56.429405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:56.429405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:56.429405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:56.429405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:56.429405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:56.429405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:56.429405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:56.429405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:56.429405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:56.429405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:56.429405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:56.429405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:56.429405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:56.429405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:23:56.430525 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:23:56.430525 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:56.430525 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:56.435964 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 300/598] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:56.435964 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b158be-1b66-41c8-9e9d-29dc37ff149c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:56.439747 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329def8f-b16b-405b-af23-2352498f1e66 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:56.486974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ec549-e5e7-46cb-8219-6cb361d7b6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:56.487844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ec549-e5e7-46cb-8219-6cb361d7b6ee None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:56.488299 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ec549-e5e7-46cb-8219-6cb361d7b6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:56.488717 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ec549-e5e7-46cb-8219-6cb361d7b6ee None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:56.489538 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ec549-e5e7-46cb-8219-6cb361d7b6ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:56.490460 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ec549-e5e7-46cb-8219-6cb361d7b6ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:56.492508 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ec549-e5e7-46cb-8219-6cb361d7b6ee None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:56.504696 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:56.508064 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 299/599] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:23:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:56.508962 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ec549-e5e7-46cb-8219-6cb361d7b6ee None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:56.539122 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b158be-1b66-41c8-9e9d-29dc37ff149c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:56.574106 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d9403-8bbb-4413-87d0-61803591751e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:56.575036 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d9403-8bbb-4413-87d0-61803591751e None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:56.575520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d9403-8bbb-4413-87d0-61803591751e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:56.575797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d9403-8bbb-4413-87d0-61803591751e None admin] PATH_INFO: `/v3/projects/71e5e82c9f454cdd91dfeac5a57968c4/users/627171bf2a2e4a21ba6f800879f8c665/roles/402c3e6fe3364285bd72784a297db11e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:56.591696 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d9403-8bbb-4413-87d0-61803591751e None admin] RBAC: Authorizing `identity:create_grant(project_id=71e5e82c9f454cdd91dfeac5a57968c4, user_id=627171bf2a2e4a21ba6f800879f8c665, role_id=402c3e6fe3364285bd72784a297db11e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:56.593844 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d9403-8bbb-4413-87d0-61803591751e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:56.619512 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ec549-e5e7-46cb-8219-6cb361d7b6ee None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:56.645789 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 301/600] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 08:23:56 2022] PUT /identity/v3/projects/71e5e82c9f454cdd91dfeac5a57968c4/users/627171bf2a2e4a21ba6f800879f8c665/roles/402c3e6fe3364285bd72784a297db11e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:23:56.647424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0332e-936b-4f49-b62c-d9d5d2609325 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:56.647776 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0332e-936b-4f49-b62c-d9d5d2609325 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:56.648056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0332e-936b-4f49-b62c-d9d5d2609325 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:56.649447 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 302/601] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:56.654493 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0332e-936b-4f49-b62c-d9d5d2609325 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:56.668081 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07316471-9ca6-4024-a8f1-2da171325246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:56.668711 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07316471-9ca6-4024-a8f1-2da171325246 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:56.669204 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07316471-9ca6-4024-a8f1-2da171325246 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:56.669650 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07316471-9ca6-4024-a8f1-2da171325246 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:56.670459 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07316471-9ca6-4024-a8f1-2da171325246 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:56.670830 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07316471-9ca6-4024-a8f1-2da171325246 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:56.672781 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07316471-9ca6-4024-a8f1-2da171325246 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:56.692947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 300/602] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:23:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:56.692947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665956b-2144-4a95-96f0-a8b62a37541a None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:56.692947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665956b-2144-4a95-96f0-a8b62a37541a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:56.692947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665956b-2144-4a95-96f0-a8b62a37541a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:56.701317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:56.716133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-c665956b-2144-4a95-96f0-a8b62a37541a None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:56.718629 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c665956b-2144-4a95-96f0-a8b62a37541a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:56.788482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0332e-936b-4f49-b62c-d9d5d2609325 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:56.813514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:56.828984 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 301/603] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:56.830007 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665956b-2144-4a95-96f0-a8b62a37541a None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:56.835867 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06b6ae-ede8-4842-bf22-369c62ede3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:56.836829 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06b6ae-ede8-4842-bf22-369c62ede3af None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:56.837332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06b6ae-ede8-4842-bf22-369c62ede3af None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:56.837782 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06b6ae-ede8-4842-bf22-369c62ede3af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:56.845676 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06b6ae-ede8-4842-bf22-369c62ede3af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:56.847591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06b6ae-ede8-4842-bf22-369c62ede3af None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:56.863233 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-8b06b6ae-ede8-4842-bf22-369c62ede3af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:23:56.863233 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:56.863233 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:56.863233 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:56.863233 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:56.863233 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:56.863233 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:56.863233 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:56.863233 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:56.863233 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:56.863233 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:56.863233 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:56.863233 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:23:56.863233 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:23:56.863233 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:23:56.864187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:23:56.864187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:56.864187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:56.864187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:56.864187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:56.864187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:56.864187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:56.864187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:56.864187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:56.864187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:56.864187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:56.864187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:56.864187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:56.864187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:56.864187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:56.864187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:56.864187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:56.865079 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:23:56.865079 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:23:56.865079 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:23:56.865079 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:23:56.865079 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:23:56.865079 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:23:56.865079 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:23:56.865079 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:56.868595 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 303/604] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:23:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:56.868595 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06b6ae-ede8-4842-bf22-369c62ede3af None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:56.940155 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665956b-2144-4a95-96f0-a8b62a37541a None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:56.980783 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67900fa9-ab84-45ae-a967-91e319b365d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:56.981442 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67900fa9-ab84-45ae-a967-91e319b365d6 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:56.981724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67900fa9-ab84-45ae-a967-91e319b365d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:56.981987 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67900fa9-ab84-45ae-a967-91e319b365d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:56.984825 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:56.991146 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67900fa9-ab84-45ae-a967-91e319b365d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:56.992854 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67900fa9-ab84-45ae-a967-91e319b365d6 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:57.010274 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06b6ae-ede8-4842-bf22-369c62ede3af None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:57.010953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-67900fa9-ab84-45ae-a967-91e319b365d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:23:57.010953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:57.010953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:57.010953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:57.010953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:57.010953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:57.010953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:57.010953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:57.010953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:57.010953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:57.010953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:57.010953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:57.010953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:23:57.010953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:23:57.010953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:23:57.011805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:23:57.011805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:57.011805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:57.011805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:57.011805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:57.011805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:57.011805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:57.011805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:57.011805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:57.011805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:57.011805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:57.011805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:57.011805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:57.011805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:57.011805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:57.011805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:57.011805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:57.012685 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:23:57.012685 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:23:57.012685 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:23:57.012685 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:23:57.012685 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:23:57.012685 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:23:57.012685 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:23:57.012685 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:57.025947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 302/605] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:23:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:57.025947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67900fa9-ab84-45ae-a967-91e319b365d6 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:57.060958 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb01cc-8b73-4641-bf18-605deea637e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:57.062049 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb01cc-8b73-4641-bf18-605deea637e3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:57.062551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb01cc-8b73-4641-bf18-605deea637e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:57.063130 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb01cc-8b73-4641-bf18-605deea637e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:57.075196 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bb01cc-8b73-4641-bf18-605deea637e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:57.081502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-85bb01cc-8b73-4641-bf18-605deea637e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:23:57.081502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:57.081502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:57.081502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:57.081502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:57.081502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:57.081502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:57.081502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:57.081502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:57.081502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:57.081502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:57.081502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:57.081502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:57.081502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:57.081502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:57.081502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:57.082696 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:57.082696 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:57.082696 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:57.082696 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:57.082696 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:57.082696 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:57.082696 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:57.082696 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:57.082696 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:57.082696 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:57.082696 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:57.082696 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:57.082696 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:57.082696 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:57.082696 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:57.082696 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:57.083520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:57.083520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:57.083520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:57.083520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:57.083520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:57.083520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:57.083520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:57.083520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:57.083520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:57.083520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:57.083520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:23:57.083520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:57.086258 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 304/606] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 08:23:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:57.087538 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb01cc-8b73-4641-bf18-605deea637e3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:57.132979 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:57.151766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67900fa9-ab84-45ae-a967-91e319b365d6 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:57.199763 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f192fb-6340-49f6-9da8-27ac4fd98d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:57.200380 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f192fb-6340-49f6-9da8-27ac4fd98d26 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:57.200660 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f192fb-6340-49f6-9da8-27ac4fd98d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:57.200932 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f192fb-6340-49f6-9da8-27ac4fd98d26 None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:57.201518 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb01cc-8b73-4641-bf18-605deea637e3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:57.201790 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f192fb-6340-49f6-9da8-27ac4fd98d26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:57.202723 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f192fb-6340-49f6-9da8-27ac4fd98d26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:57.204168 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f192fb-6340-49f6-9da8-27ac4fd98d26 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:57.223102 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 303/607] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:23:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:57.224312 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f192fb-6340-49f6-9da8-27ac4fd98d26 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:57.243474 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4a9a9-46f4-4132-974c-ef37122afaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:57.244089 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4a9a9-46f4-4132-974c-ef37122afaaf None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:57.244368 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4a9a9-46f4-4132-974c-ef37122afaaf None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:57.244641 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4a9a9-46f4-4132-974c-ef37122afaaf None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:57.245706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4a9a9-46f4-4132-974c-ef37122afaaf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:57.246016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4a9a9-46f4-4132-974c-ef37122afaaf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:57.247970 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4a9a9-46f4-4132-974c-ef37122afaaf None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:57.266545 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 305/608] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:23:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:57.271908 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4a9a9-46f4-4132-974c-ef37122afaaf None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:57.288943 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:57.337562 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f192fb-6340-49f6-9da8-27ac4fd98d26 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:57.374014 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b969b91-27e5-4ca2-9df3-73c05c437ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:57.374635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b969b91-27e5-4ca2-9df3-73c05c437ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:57.374934 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b969b91-27e5-4ca2-9df3-73c05c437ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:57.375199 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b969b91-27e5-4ca2-9df3-73c05c437ed5 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:57.375863 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b969b91-27e5-4ca2-9df3-73c05c437ed5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:57.376162 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b969b91-27e5-4ca2-9df3-73c05c437ed5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:57.377552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b969b91-27e5-4ca2-9df3-73c05c437ed5 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:57.386027 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 304/609] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 08:23:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:57.386681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b969b91-27e5-4ca2-9df3-73c05c437ed5 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:57.414604 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:57.422915 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4a9a9-46f4-4132-974c-ef37122afaaf None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:57.463702 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d1a4e-370f-4fb1-b3c0-a1c945fb6d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:57.464610 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d1a4e-370f-4fb1-b3c0-a1c945fb6d68 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:57.465036 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d1a4e-370f-4fb1-b3c0-a1c945fb6d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:57.465592 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d1a4e-370f-4fb1-b3c0-a1c945fb6d68 None admin] PATH_INFO: `/v3/projects/71e5e82c9f454cdd91dfeac5a57968c4/users/3262fa2d9c0d4d8998c9697994b48fef/roles/402c3e6fe3364285bd72784a297db11e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:57.481595 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d1a4e-370f-4fb1-b3c0-a1c945fb6d68 None admin] RBAC: Authorizing `identity:create_grant(project_id=71e5e82c9f454cdd91dfeac5a57968c4, user_id=3262fa2d9c0d4d8998c9697994b48fef, role_id=402c3e6fe3364285bd72784a297db11e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:57.483553 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d1a4e-370f-4fb1-b3c0-a1c945fb6d68 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:57.506114 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b969b91-27e5-4ca2-9df3-73c05c437ed5 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:57.522583 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:57.540293 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8597498-68b0-4ba4-b961-52ce1844dbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:57.540953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8597498-68b0-4ba4-b961-52ce1844dbe0 None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:57.541268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8597498-68b0-4ba4-b961-52ce1844dbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:57.541546 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8597498-68b0-4ba4-b961-52ce1844dbe0 None admin] PATH_INFO: `/v3/projects/71e5e82c9f454cdd91dfeac5a57968c4/users/2c3b8b43026c40209bfc42acd2830610/roles/402c3e6fe3364285bd72784a297db11e` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:57.555832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8597498-68b0-4ba4-b961-52ce1844dbe0 None admin] RBAC: Authorizing `identity:create_grant(project_id=71e5e82c9f454cdd91dfeac5a57968c4, user_id=2c3b8b43026c40209bfc42acd2830610, role_id=402c3e6fe3364285bd72784a297db11e)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:57.557290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8597498-68b0-4ba4-b961-52ce1844dbe0 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:57.570773 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 306/610] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 08:23:57 2022] PUT /identity/v3/projects/71e5e82c9f454cdd91dfeac5a57968c4/users/3262fa2d9c0d4d8998c9697994b48fef/roles/402c3e6fe3364285bd72784a297db11e => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:23:57.572009 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881f8c3-237c-4708-ab92-9a69b204034c None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:57.572290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881f8c3-237c-4708-ab92-9a69b204034c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:57.572559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881f8c3-237c-4708-ab92-9a69b204034c None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:57.573960 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 307/611] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:57.574512 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881f8c3-237c-4708-ab92-9a69b204034c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:57.637201 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 305/612] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 08:23:57 2022] PUT /identity/v3/projects/71e5e82c9f454cdd91dfeac5a57968c4/users/2c3b8b43026c40209bfc42acd2830610/roles/402c3e6fe3364285bd72784a297db11e => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:23:57.637201 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877927cc-fe3e-4218-b185-4088809a7646 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:57.637201 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877927cc-fe3e-4218-b185-4088809a7646 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:57.637201 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877927cc-fe3e-4218-b185-4088809a7646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:57.676188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-877927cc-fe3e-4218-b185-4088809a7646 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:57.677572 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-877927cc-fe3e-4218-b185-4088809a7646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:57.754297 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881f8c3-237c-4708-ab92-9a69b204034c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:57.799867 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d42ff7-4fbe-4622-af0f-392aae255ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:57.800514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d42ff7-4fbe-4622-af0f-392aae255ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:57.800784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d42ff7-4fbe-4622-af0f-392aae255ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:57.801055 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d42ff7-4fbe-4622-af0f-392aae255ac7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:57.805996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 306/613] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:57.814815 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d42ff7-4fbe-4622-af0f-392aae255ac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:57.814815 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d42ff7-4fbe-4622-af0f-392aae255ac7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:57.824256 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:57.831173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-29d42ff7-4fbe-4622-af0f-392aae255ac7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:57.831173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:57.831173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:57.831173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:57.831173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:57.831173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:57.831173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:57.831173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:57.831173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:57.831173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:57.831173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:57.831173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:57.831173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:23:57.831173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:23:57.831173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:23:57.831173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:23:57.832520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:57.832520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:57.832520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:57.832520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:57.832520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:57.832520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:57.832520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:57.832520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:57.832520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:57.832520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:57.832520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:57.832520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:57.832520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:57.832520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:57.832520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:57.832520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:57.832520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:23:57.843465 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:23:57.843465 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:57.843465 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:57.843465 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 308/614] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:57.843843 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877927cc-fe3e-4218-b185-4088809a7646 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:57.844980 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d42ff7-4fbe-4622-af0f-392aae255ac7 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:57.963644 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877927cc-fe3e-4218-b185-4088809a7646 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:57.978162 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d42ff7-4fbe-4622-af0f-392aae255ac7 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:57.989532 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:58.001297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34932b4-4071-4415-af2f-ebed3a30f0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:58.001836 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34932b4-4071-4415-af2f-ebed3a30f0cb None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:58.002211 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34932b4-4071-4415-af2f-ebed3a30f0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:58.002583 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34932b4-4071-4415-af2f-ebed3a30f0cb None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:58.003316 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34932b4-4071-4415-af2f-ebed3a30f0cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:58.003710 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34932b4-4071-4415-af2f-ebed3a30f0cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:58.005368 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34932b4-4071-4415-af2f-ebed3a30f0cb None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:58.017375 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 307/615] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:23:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:58.021435 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b3e4c-017b-43da-8c49-49fa2b7cd4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:58.022243 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b3e4c-017b-43da-8c49-49fa2b7cd4d8 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:58.022666 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b3e4c-017b-43da-8c49-49fa2b7cd4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:58.023081 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b3e4c-017b-43da-8c49-49fa2b7cd4d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:58.023410 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34932b4-4071-4415-af2f-ebed3a30f0cb None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:58.036140 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5b3e4c-017b-43da-8c49-49fa2b7cd4d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:58.043502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-8f5b3e4c-017b-43da-8c49-49fa2b7cd4d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:23:58.043502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:58.043502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:58.043502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:58.043502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:58.043502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:58.043502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:58.043502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:58.043502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:58.043502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:58.043502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:58.043502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:58.043502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:58.043502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:58.043502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:58.043502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:58.044609 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:58.044609 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:58.044609 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:58.044609 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:58.044609 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:58.044609 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:58.044609 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:58.044609 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:58.044609 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:58.044609 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:58.044609 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:58.044609 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:58.044609 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:58.044609 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:58.044609 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:58.044609 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:58.045657 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:58.045657 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:58.045657 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:58.045657 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:58.045657 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:58.045657 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:58.045657 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:58.045657 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:58.045657 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:58.045657 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:58.045657 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:23:58.045657 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:58.048927 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 309/616] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 08:23:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:58.058127 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b3e4c-017b-43da-8c49-49fa2b7cd4d8 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:58.135070 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34932b4-4071-4415-af2f-ebed3a30f0cb None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:58.169844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa38f60-92fb-4b6c-87b0-eeee1b6496d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:58.170656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa38f60-92fb-4b6c-87b0-eeee1b6496d6 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:58.171703 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa38f60-92fb-4b6c-87b0-eeee1b6496d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:58.172063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa38f60-92fb-4b6c-87b0-eeee1b6496d6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:58.179226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:58.180054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa38f60-92fb-4b6c-87b0-eeee1b6496d6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:58.181706 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa38f60-92fb-4b6c-87b0-eeee1b6496d6 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:58.207322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-5fa38f60-92fb-4b6c-87b0-eeee1b6496d6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 08:23:58.207322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:58.207322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:58.207322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:58.207322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:58.207322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:58.207322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:58.207322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:58.207322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:58.207322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:58.207322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:58.207322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:58.207322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:23:58.207322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:23:58.207322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:23:58.207322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:23:58.208486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:58.208486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:58.208486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:23:58.208486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:23:58.208486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:23:58.208486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:23:58.208486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:58.208486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:58.208486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:58.208486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:58.208486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:58.208486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:58.208486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:58.208486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:58.208486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:58.208486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:58.208486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:58.208486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:58.209844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:58.209844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:58.209844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:23:58.209844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:23:58.209844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:23:58.209844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:23:58.209844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:23:58.209844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:23:58.209844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 08:23:58.209844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:58.212619 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 308/617] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:23:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:23:58.219960 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa38f60-92fb-4b6c-87b0-eeee1b6496d6 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:58.259338 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b3e4c-017b-43da-8c49-49fa2b7cd4d8 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:58.265507 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:58.331319 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee6580-a023-4498-b370-4ec316c9bc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:58.332351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee6580-a023-4498-b370-4ec316c9bc5e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:58.332724 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee6580-a023-4498-b370-4ec316c9bc5e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:58.333088 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee6580-a023-4498-b370-4ec316c9bc5e None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:58.333880 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbee6580-a023-4498-b370-4ec316c9bc5e None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:58.334272 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbee6580-a023-4498-b370-4ec316c9bc5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:58.336981 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbee6580-a023-4498-b370-4ec316c9bc5e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:58.354508 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 310/618] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 08:23:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:58.360234 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee6580-a023-4498-b370-4ec316c9bc5e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:58.415429 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa38f60-92fb-4b6c-87b0-eeee1b6496d6 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:58.449066 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648179a-eabd-4b39-a478-18a9cd050f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:58.449795 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648179a-eabd-4b39-a478-18a9cd050f35 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:58.450072 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648179a-eabd-4b39-a478-18a9cd050f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:58.450341 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648179a-eabd-4b39-a478-18a9cd050f35 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:58.451068 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3648179a-eabd-4b39-a478-18a9cd050f35 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:58.451367 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3648179a-eabd-4b39-a478-18a9cd050f35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:58.453324 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3648179a-eabd-4b39-a478-18a9cd050f35 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:58.473266 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 309/619] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Aug 8 08:23:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:58.478603 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648179a-eabd-4b39-a478-18a9cd050f35 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:58.551287 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee6580-a023-4498-b370-4ec316c9bc5e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:58.562195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:58.590711 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:58.599043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648179a-eabd-4b39-a478-18a9cd050f35 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:58.617957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a06a39-d06e-4772-98c9-e0176403eb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:58.618598 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a06a39-d06e-4772-98c9-e0176403eb17 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:58.618873 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a06a39-d06e-4772-98c9-e0176403eb17 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:58.619139 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a06a39-d06e-4772-98c9-e0176403eb17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:58.641633 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a06a39-d06e-4772-98c9-e0176403eb17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:58.643110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a06a39-d06e-4772-98c9-e0176403eb17 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:58.670421 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6160db-46da-47d5-a5de-e92119bc330f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:58.671895 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6160db-46da-47d5-a5de-e92119bc330f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:58.672174 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6160db-46da-47d5-a5de-e92119bc330f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:58.672441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6160db-46da-47d5-a5de-e92119bc330f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:58.673336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-d2a06a39-d06e-4772-98c9-e0176403eb17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:58.673336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:58.673336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:58.673336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:58.673336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:58.673336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:58.673336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:58.673336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:58.673336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:58.673336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:58.673336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:58.673336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:58.673336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:23:58.673336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:23:58.673336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:23:58.673336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:23:58.674270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:58.674270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:58.674270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:58.674270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:58.674270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:58.674270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:58.674270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:58.674270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:58.674270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:58.674270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:58.674270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:58.674270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:58.674270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:58.674270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:58.674270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:58.674270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:58.674270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:23:58.676315 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:23:58.676315 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:58.676315 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:58.676315 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 311/620] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:23:58.682180 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a06a39-d06e-4772-98c9-e0176403eb17 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:58.694269 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6160db-46da-47d5-a5de-e92119bc330f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:58.695725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6160db-46da-47d5-a5de-e92119bc330f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:58.720953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-8d6160db-46da-47d5-a5de-e92119bc330f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:58.720953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:58.720953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:58.720953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:58.720953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:58.720953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:58.720953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:58.720953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:58.720953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:58.720953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:58.720953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:58.720953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:58.720953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:23:58.720953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:23:58.720953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:23:58.720953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:23:58.721980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:58.721980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:58.721980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:58.721980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:58.721980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:58.721980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:58.721980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:58.721980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:58.721980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:58.721980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:58.721980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:58.721980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:58.721980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:58.721980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:58.721980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:58.721980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:58.721980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:23:58.722864 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:23:58.722864 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:23:58.722864 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:58.727523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 310/621] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:58.734791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6160db-46da-47d5-a5de-e92119bc330f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:58.883552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6160db-46da-47d5-a5de-e92119bc330f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:58.886579 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a06a39-d06e-4772-98c9-e0176403eb17 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:58.910226 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:58.919037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ca264d-96db-4e98-b8cf-0947024f635e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:58.919646 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca264d-96db-4e98-b8cf-0947024f635e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:58.919923 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca264d-96db-4e98-b8cf-0947024f635e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:58.920197 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca264d-96db-4e98-b8cf-0947024f635e None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:58.921398 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ca264d-96db-4e98-b8cf-0947024f635e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:58.921918 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ca264d-96db-4e98-b8cf-0947024f635e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:58.923384 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ca264d-96db-4e98-b8cf-0947024f635e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:58.924046 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b60b167-f2f0-468a-84c4-b54d2d0894c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:58.924659 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60b167-f2f0-468a-84c4-b54d2d0894c4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:58.924941 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60b167-f2f0-468a-84c4-b54d2d0894c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:58.925261 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60b167-f2f0-468a-84c4-b54d2d0894c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:58.925912 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60b167-f2f0-468a-84c4-b54d2d0894c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:58.926216 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60b167-f2f0-468a-84c4-b54d2d0894c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:58.927584 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60b167-f2f0-468a-84c4-b54d2d0894c4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:58.936190 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 311/622] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:23:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:58.938406 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1869ad0-fdb8-4805-873e-1d120f1f7178 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:58.938692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1869ad0-fdb8-4805-873e-1d120f1f7178 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:58.938961 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1869ad0-fdb8-4805-873e-1d120f1f7178 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:58.940447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 312/623] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:23:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:23:58.945481 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 312/624] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:23:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:58.946301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1869ad0-fdb8-4805-873e-1d120f1f7178 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:58.946943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a37f5-01ec-4ff8-88c6-349ac0c23034 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:58.947241 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a37f5-01ec-4ff8-88c6-349ac0c23034 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:58.947554 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a37f5-01ec-4ff8-88c6-349ac0c23034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:58.967949 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-6a7a37f5-01ec-4ff8-88c6-349ac0c23034 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:23:58.969317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6a7a37f5-01ec-4ff8-88c6-349ac0c23034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:23:59.001634 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:59.057405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 313/625] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:23:59.058325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a37f5-01ec-4ff8-88c6-349ac0c23034 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:59.072132 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1869ad0-fdb8-4805-873e-1d120f1f7178 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:59.126099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27133eb7-c39c-441a-9dde-9492265c1986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:59.127254 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27133eb7-c39c-441a-9dde-9492265c1986 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:59.128166 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27133eb7-c39c-441a-9dde-9492265c1986 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:59.128743 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27133eb7-c39c-441a-9dde-9492265c1986 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:59.138248 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27133eb7-c39c-441a-9dde-9492265c1986 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:59.142100 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27133eb7-c39c-441a-9dde-9492265c1986 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:59.157343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-27133eb7-c39c-441a-9dde-9492265c1986 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:23:59.157343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:59.157343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:59.157343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:59.157343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:59.157343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:59.157343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:59.157343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:59.157343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:59.157343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:59.157343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:59.157343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:59.157343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:23:59.157343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:23:59.157343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:23:59.158447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:23:59.158447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:59.158447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:59.158447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:59.158447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:59.158447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:59.158447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:59.158447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:59.158447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:59.158447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:59.158447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:59.158447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:59.158447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:59.158447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:59.158447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:59.158447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:59.158447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:59.159514 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:23:59.159514 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:23:59.159514 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:23:59.159514 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:23:59.159514 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:23:59.159514 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:23:59.159514 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:23:59.159514 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:59.162040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 313/626] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:23:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:59.166873 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27133eb7-c39c-441a-9dde-9492265c1986 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:59.190819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a37f5-01ec-4ff8-88c6-349ac0c23034 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:59.229022 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:59.231893 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332de0bd-90f5-487f-aaf5-c37f4af5255a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:59.233007 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332de0bd-90f5-487f-aaf5-c37f4af5255a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:59.233954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332de0bd-90f5-487f-aaf5-c37f4af5255a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:59.234518 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332de0bd-90f5-487f-aaf5-c37f4af5255a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:59.244439 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332de0bd-90f5-487f-aaf5-c37f4af5255a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:59.246807 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332de0bd-90f5-487f-aaf5-c37f4af5255a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:59.269070 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-332de0bd-90f5-487f-aaf5-c37f4af5255a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 08:23:59.269070 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:59.269070 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:59.269070 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:59.269070 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:59.269070 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:59.269070 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:59.269070 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:59.269070 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:59.269070 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:59.269070 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:59.269070 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:59.269070 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:23:59.269070 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:23:59.269070 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:23:59.269070 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:23:59.270277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:59.270277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:59.270277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:23:59.270277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:23:59.270277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:23:59.270277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:23:59.270277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:59.270277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:59.270277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:59.270277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:59.270277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:59.270277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:59.270277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:59.270277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:59.270277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:59.270277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:59.270277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:59.270277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:59.271452 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:59.271452 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:59.271452 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:23:59.271452 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:23:59.271452 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:23:59.271452 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:23:59.271452 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:23:59.271452 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:23:59.271452 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 08:23:59.271452 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:23:59.274423 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 314/627] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Aug 8 08:23:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:59.276092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332de0bd-90f5-487f-aaf5-c37f4af5255a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:59.292286 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27133eb7-c39c-441a-9dde-9492265c1986 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:59.310610 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:59.335584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83100061-d7c6-4157-9344-6ab999c6a5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:59.336250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83100061-d7c6-4157-9344-6ab999c6a5ad None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:59.336531 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83100061-d7c6-4157-9344-6ab999c6a5ad None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:59.336813 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83100061-d7c6-4157-9344-6ab999c6a5ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:59.351693 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83100061-d7c6-4157-9344-6ab999c6a5ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:23:59.357400 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-83100061-d7c6-4157-9344-6ab999c6a5ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:23:59.357400 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:23:59.357400 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:23:59.357400 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:23:59.357400 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:23:59.357400 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:23:59.357400 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:23:59.357400 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:23:59.357400 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:23:59.357400 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:23:59.357400 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:23:59.357400 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:23:59.357400 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:23:59.357400 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:23:59.357400 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:23:59.357400 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:23:59.358987 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:23:59.358987 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:23:59.358987 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:23:59.358987 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:23:59.358987 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:23:59.358987 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:23:59.358987 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:23:59.358987 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:23:59.358987 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:23:59.358987 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:23:59.358987 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:23:59.358987 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:23:59.358987 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:23:59.358987 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:23:59.358987 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:23:59.358987 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:23:59.359809 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:23:59.359809 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:23:59.359809 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:23:59.359809 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:23:59.359809 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:23:59.359809 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:23:59.359809 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:23:59.359809 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:23:59.359809 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:23:59.359809 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:23:59.359809 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:23:59.359809 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:23:59.364555 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 314/628] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 08:23:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:23:59.365426 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83100061-d7c6-4157-9344-6ab999c6a5ad None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:59.399808 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332de0bd-90f5-487f-aaf5-c37f4af5255a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:59.444631 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36fc81e-f238-447f-abc9-6f1ac6b42011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:59.445549 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fc81e-f238-447f-abc9-6f1ac6b42011 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:59.446043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fc81e-f238-447f-abc9-6f1ac6b42011 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:59.446455 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fc81e-f238-447f-abc9-6f1ac6b42011 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:59.447282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36fc81e-f238-447f-abc9-6f1ac6b42011 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:59.447741 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36fc81e-f238-447f-abc9-6f1ac6b42011 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:59.449349 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36fc81e-f238-447f-abc9-6f1ac6b42011 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:59.463305 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 315/629] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:23:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:59.464418 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36fc81e-f238-447f-abc9-6f1ac6b42011 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:59.501005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83100061-d7c6-4157-9344-6ab999c6a5ad None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:59.510065 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:59.545887 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfa73a-c381-4af2-abc7-b8fe7a1ca88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:59.546763 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfa73a-c381-4af2-abc7-b8fe7a1ca88d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:59.547155 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfa73a-c381-4af2-abc7-b8fe7a1ca88d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:59.547540 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfa73a-c381-4af2-abc7-b8fe7a1ca88d None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:59.548960 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfa73a-c381-4af2-abc7-b8fe7a1ca88d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:59.549437 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfa73a-c381-4af2-abc7-b8fe7a1ca88d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:59.551448 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfa73a-c381-4af2-abc7-b8fe7a1ca88d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:59.559269 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:59.573864 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 315/630] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Aug 8 08:23:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:59.574954 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfa73a-c381-4af2-abc7-b8fe7a1ca88d None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:59.584395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36fc81e-f238-447f-abc9-6f1ac6b42011 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:59.635243 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b3c8c-6d80-4189-8110-d9325aeaa013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:59.636042 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b3c8c-6d80-4189-8110-d9325aeaa013 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:59.636449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b3c8c-6d80-4189-8110-d9325aeaa013 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:59.637100 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b3c8c-6d80-4189-8110-d9325aeaa013 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:59.637913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b3c8c-6d80-4189-8110-d9325aeaa013 None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:23:59.638583 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b3c8c-6d80-4189-8110-d9325aeaa013 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:59.641379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b3c8c-6d80-4189-8110-d9325aeaa013 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:59.657732 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 316/631] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 08:23:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:23:59.659521 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b3c8c-6d80-4189-8110-d9325aeaa013 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:59.692251 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfa73a-c381-4af2-abc7-b8fe7a1ca88d None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:59.738090 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812382ae-9b2d-42c3-aa2e-70752d8188cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:59.739847 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812382ae-9b2d-42c3-aa2e-70752d8188cb None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:59.740745 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812382ae-9b2d-42c3-aa2e-70752d8188cb None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:59.741507 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812382ae-9b2d-42c3-aa2e-70752d8188cb None admin] PATH_INFO: `/v3/projects/71e5e82c9f454cdd91dfeac5a57968c4/users/e69c0e02ad944f2a85fc5318018e5cb0/roles/402c3e6fe3364285bd72784a297db11e` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:59.763252 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812382ae-9b2d-42c3-aa2e-70752d8188cb None admin] RBAC: Authorizing `identity:create_grant(project_id=71e5e82c9f454cdd91dfeac5a57968c4, user_id=e69c0e02ad944f2a85fc5318018e5cb0, role_id=402c3e6fe3364285bd72784a297db11e)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:59.766317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812382ae-9b2d-42c3-aa2e-70752d8188cb None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:59.783507 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b3c8c-6d80-4189-8110-d9325aeaa013 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:23:59.819464 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264cabad-bf07-4c07-b852-5851787d2895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:23:59.819997 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cabad-bf07-4c07-b852-5851787d2895 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:23:59.820302 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cabad-bf07-4c07-b852-5851787d2895 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:23:59.820613 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cabad-bf07-4c07-b852-5851787d2895 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:23:59.836169 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 316/632] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 08:23:59 2022] PUT /identity/v3/projects/71e5e82c9f454cdd91dfeac5a57968c4/users/e69c0e02ad944f2a85fc5318018e5cb0/roles/402c3e6fe3364285bd72784a297db11e => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:23:59.838276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:59.839195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812382ae-9b2d-42c3-aa2e-70752d8188cb None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:23:59.862873 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:23:59.899605 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264cabad-bf07-4c07-b852-5851787d2895 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:23:59.901960 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264cabad-bf07-4c07-b852-5851787d2895 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:23:59.986867 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812382ae-9b2d-42c3-aa2e-70752d8188cb None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:00.024948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 317/633] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:23:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:24:00.026605 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb65e3-da01-44a3-94b2-5a0ae6887b92 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:00.027094 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb65e3-da01-44a3-94b2-5a0ae6887b92 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:00.027587 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb65e3-da01-44a3-94b2-5a0ae6887b92 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:00.029321 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 318/634] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:00.031358 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb65e3-da01-44a3-94b2-5a0ae6887b92 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:00.040187 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63737fda-0b70-47b4-891e-1b0069b65a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:00.041856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63737fda-0b70-47b4-891e-1b0069b65a3f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:00.041856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63737fda-0b70-47b4-891e-1b0069b65a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:00.042108 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63737fda-0b70-47b4-891e-1b0069b65a3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:00.055787 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63737fda-0b70-47b4-891e-1b0069b65a3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:00.057408 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63737fda-0b70-47b4-891e-1b0069b65a3f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:00.073832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-63737fda-0b70-47b4-891e-1b0069b65a3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:00.073832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:00.073832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:00.073832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:00.073832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:00.073832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:00.073832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:00.073832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:00.073832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:00.073832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:00.073832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:00.073832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:00.073832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:00.073832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:00.073832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:00.073832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:00.074977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:00.074977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:00.074977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:00.074977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:00.074977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:00.074977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:00.074977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:00.074977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:00.074977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:00.074977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:00.074977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:00.074977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:00.074977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:00.074977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:00.074977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:00.074977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:00.074977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:00.076063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:00.076063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:00.076063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:00.077378 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 317/635] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:23:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:00.078716 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f4162-e0c5-4f17-9061-3da046c6b05f None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:00.079113 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f4162-e0c5-4f17-9061-3da046c6b05f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:00.079486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f4162-e0c5-4f17-9061-3da046c6b05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:00.093701 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:00.099293 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-0b8f4162-e0c5-4f17-9061-3da046c6b05f None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:00.100669 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0b8f4162-e0c5-4f17-9061-3da046c6b05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:00.155789 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb65e3-da01-44a3-94b2-5a0ae6887b92 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:00.167241 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:00.175957 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 318/636] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:24:00.176856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f4162-e0c5-4f17-9061-3da046c6b05f None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:00.201441 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd8a39c-930a-4b8c-8883-ee9f9b8a29d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:00.202423 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8a39c-930a-4b8c-8883-ee9f9b8a29d2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:00.203196 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8a39c-930a-4b8c-8883-ee9f9b8a29d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:00.203466 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8a39c-930a-4b8c-8883-ee9f9b8a29d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:00.214867 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd8a39c-930a-4b8c-8883-ee9f9b8a29d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:00.216324 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd8a39c-930a-4b8c-8883-ee9f9b8a29d2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:00.232582 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-bbd8a39c-930a-4b8c-8883-ee9f9b8a29d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:00.232582 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:00.232582 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:00.232582 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:00.232582 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:00.232582 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:00.232582 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:00.232582 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:00.232582 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:00.232582 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:00.232582 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:00.232582 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:00.232582 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:00.232582 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:00.232582 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:00.232582 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:00.234531 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:00.234531 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:00.234531 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:00.234531 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:00.234531 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:00.234531 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:00.234531 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:00.234531 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:00.234531 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:00.234531 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:00.234531 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:00.234531 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:00.234531 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:00.234531 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:00.234531 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:00.234531 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:00.234531 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:00.235596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:00.235596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:00.235596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:00.235596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 319/637] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:00.235873 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be7f3f-081d-494a-b2b5-6f00a0218edd None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:00.236084 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be7f3f-081d-494a-b2b5-6f00a0218edd None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:00.236362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be7f3f-081d-494a-b2b5-6f00a0218edd None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:00.241500 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 320/638] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:00.241500 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be7f3f-081d-494a-b2b5-6f00a0218edd None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:00.287388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f4162-e0c5-4f17-9061-3da046c6b05f None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:00.321395 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465f2da-c526-4ad2-8315-b636e9f26d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:00.321996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465f2da-c526-4ad2-8315-b636e9f26d79 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:00.322272 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465f2da-c526-4ad2-8315-b636e9f26d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:00.322540 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465f2da-c526-4ad2-8315-b636e9f26d79 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:00.323243 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b465f2da-c526-4ad2-8315-b636e9f26d79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:00.323481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b465f2da-c526-4ad2-8315-b636e9f26d79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:00.324885 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b465f2da-c526-4ad2-8315-b636e9f26d79 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:00.336396 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 319/639] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:00.337226 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465f2da-c526-4ad2-8315-b636e9f26d79 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:00.348497 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:00.353969 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be7f3f-081d-494a-b2b5-6f00a0218edd None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:00.393580 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1933d-bd86-43a7-a4bf-4a7278924bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:00.394566 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1933d-bd86-43a7-a4bf-4a7278924bff None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:00.394988 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1933d-bd86-43a7-a4bf-4a7278924bff None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:00.395400 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1933d-bd86-43a7-a4bf-4a7278924bff None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:00.400271 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:00.408594 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca1933d-bd86-43a7-a4bf-4a7278924bff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:24:00.413729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-8ca1933d-bd86-43a7-a4bf-4a7278924bff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:24:00.413729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:00.413729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:00.413729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:00.413729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:00.413729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:00.413729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:00.413729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:00.413729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:00.413729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:00.413729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:00.413729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:00.413729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:24:00.413729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:24:00.413729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:24:00.413729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:24:00.414827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:24:00.414827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:24:00.414827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:24:00.414827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:24:00.414827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:00.414827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:00.414827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:00.414827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:00.414827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:00.414827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:00.414827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:00.414827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:00.414827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:00.414827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:00.414827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:00.414827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:00.415860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:00.415860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:00.415860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:00.415860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:00.415860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:24:00.415860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:24:00.415860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:24:00.415860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:24:00.415860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:24:00.415860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:24:00.415860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:24:00.415860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:00.418451 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 321/640] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 08:24:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:00.419786 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734e272-36ef-412a-9270-6f5e9ee09566 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:00.420221 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734e272-36ef-412a-9270-6f5e9ee09566 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:00.420620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734e272-36ef-412a-9270-6f5e9ee09566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:00.439134 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-1734e272-36ef-412a-9270-6f5e9ee09566 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:00.440344 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1734e272-36ef-412a-9270-6f5e9ee09566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:00.455145 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465f2da-c526-4ad2-8315-b636e9f26d79 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:00.490218 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca3b06-579c-47dc-b176-6d0d5e64fd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:00.490827 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca3b06-579c-47dc-b176-6d0d5e64fd39 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:00.491107 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca3b06-579c-47dc-b176-6d0d5e64fd39 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:00.491401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca3b06-579c-47dc-b176-6d0d5e64fd39 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:00.492012 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca3b06-579c-47dc-b176-6d0d5e64fd39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:00.492313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca3b06-579c-47dc-b176-6d0d5e64fd39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:00.493718 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca3b06-579c-47dc-b176-6d0d5e64fd39 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:00.507382 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 320/641] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:00.508067 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca3b06-579c-47dc-b176-6d0d5e64fd39 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:00.521213 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 322/642] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:24:00.521213 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734e272-36ef-412a-9270-6f5e9ee09566 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:00.617891 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca3b06-579c-47dc-b176-6d0d5e64fd39 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:00.618559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:00.632533 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734e272-36ef-412a-9270-6f5e9ee09566 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:00.636893 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:00.654897 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80286715-828a-47fe-ac27-8d78369975df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:00.656594 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80286715-828a-47fe-ac27-8d78369975df None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:00.656594 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80286715-828a-47fe-ac27-8d78369975df None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:00.656774 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80286715-828a-47fe-ac27-8d78369975df None admin] PATH_INFO: `/v3/users/glance` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:00.664071 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80286715-828a-47fe-ac27-8d78369975df None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:00.667150 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80286715-828a-47fe-ac27-8d78369975df None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:00.673206 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64197495-0869-4f86-b417-7c8899d5c54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:00.673820 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64197495-0869-4f86-b417-7c8899d5c54a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:00.674100 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64197495-0869-4f86-b417-7c8899d5c54a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:00.674371 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64197495-0869-4f86-b417-7c8899d5c54a None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:00.675039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64197495-0869-4f86-b417-7c8899d5c54a None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:00.675347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64197495-0869-4f86-b417-7c8899d5c54a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:00.676699 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64197495-0869-4f86-b417-7c8899d5c54a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:00.677722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-80286715-828a-47fe-ac27-8d78369975df None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 08:24:00.677722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:00.677722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:00.677722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:00.677722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:00.677722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:00.677722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:00.677722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:00.677722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:00.677722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:00.677722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:00.677722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:00.677722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:24:00.677722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:24:00.677722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:24:00.677722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:24:00.679029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:00.679029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:00.679029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:24:00.679029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:00.679029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:24:00.679029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:00.679029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:00.679029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:00.679029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:00.679029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:00.679029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:00.679029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:00.679029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:00.679029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:00.679029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:00.679029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:00.679029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:00.679029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:00.679974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:00.679974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:00.679974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:24:00.679974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:24:00.679974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:24:00.679974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:24:00.679974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:24:00.679974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:24:00.679974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 08:24:00.679974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:00.684877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 321/643] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:24:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:00.685739 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80286715-828a-47fe-ac27-8d78369975df None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:00.687129 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 323/644] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 08:24:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:00.687700 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64197495-0869-4f86-b417-7c8899d5c54a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:00.792043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80286715-828a-47fe-ac27-8d78369975df None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:00.792586 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64197495-0869-4f86-b417-7c8899d5c54a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:00.827735 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4978f11-1e59-4b82-8c05-91ebd91d5cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:00.828358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4978f11-1e59-4b82-8c05-91ebd91d5cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:00.828636 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4978f11-1e59-4b82-8c05-91ebd91d5cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:00.828908 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4978f11-1e59-4b82-8c05-91ebd91d5cc2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:00.831164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c5a34-f4c6-434d-9db8-1a62624c1e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:00.832518 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c5a34-f4c6-434d-9db8-1a62624c1e6d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:00.832888 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c5a34-f4c6-434d-9db8-1a62624c1e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:00.833290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c5a34-f4c6-434d-9db8-1a62624c1e6d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:00.835870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4978f11-1e59-4b82-8c05-91ebd91d5cc2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:00.837336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4978f11-1e59-4b82-8c05-91ebd91d5cc2 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:00.845748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904c5a34-f4c6-434d-9db8-1a62624c1e6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:24:00.849133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-e4978f11-1e59-4b82-8c05-91ebd91d5cc2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:00.849133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:00.849133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:00.849133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:00.849133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:00.849133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:00.849133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:00.849133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:00.849133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:00.849133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:00.849133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:00.849133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:00.849133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:24:00.849133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:24:00.849133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:24:00.850195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:24:00.850195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:00.850195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:00.850195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:00.850195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:00.850195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:00.850195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:00.850195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:00.850195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:00.850195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:00.850195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:00.850195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:00.850195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:00.850195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:00.850195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:00.850195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:00.850195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:00.851497 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:24:00.851497 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:24:00.851497 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:24:00.851497 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:24:00.851497 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:24:00.851497 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:24:00.851497 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:00.851497 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:00.853444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 322/645] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:24:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:00.854345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-904c5a34-f4c6-434d-9db8-1a62624c1e6d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:24:00.854345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:00.854345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:00.854345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:00.854345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:00.854345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:00.854345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:00.854345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:00.854345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:00.854345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:00.854345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:00.854345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:00.854345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:24:00.854345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:24:00.854345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:24:00.854345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:24:00.855332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:24:00.855332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:24:00.855332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:24:00.855332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:24:00.855332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:00.855332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:00.855332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:00.855332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:00.855332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:00.855332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:00.855332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:00.855332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:00.855332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:00.855332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:00.855332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:00.855332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:00.856332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:00.856332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:00.856332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:00.856332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:00.856332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:24:00.856332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:24:00.856332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:24:00.856332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:24:00.856332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:24:00.856332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:24:00.856332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:24:00.856332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:00.856332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 324/646] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 08:24:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:00.857078 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c5a34-f4c6-434d-9db8-1a62624c1e6d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:00.857316 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4978f11-1e59-4b82-8c05-91ebd91d5cc2 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:00.870643 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:00.898108 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:00.960206 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c5a34-f4c6-434d-9db8-1a62624c1e6d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:00.964811 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4978f11-1e59-4b82-8c05-91ebd91d5cc2 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:00.996520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d7dc0-f907-4b32-8ece-bb6d7d9eb1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:00.997341 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d7dc0-f907-4b32-8ece-bb6d7d9eb1c5 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:00.997620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d7dc0-f907-4b32-8ece-bb6d7d9eb1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:00.997917 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d7dc0-f907-4b32-8ece-bb6d7d9eb1c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:01.001050 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f45af7-423e-44b8-b54f-eaebde2bac7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:01.001696 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f45af7-423e-44b8-b54f-eaebde2bac7e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:01.001980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f45af7-423e-44b8-b54f-eaebde2bac7e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:01.002257 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f45af7-423e-44b8-b54f-eaebde2bac7e None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:01.002985 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f45af7-423e-44b8-b54f-eaebde2bac7e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:01.003288 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f45af7-423e-44b8-b54f-eaebde2bac7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:01.004693 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f45af7-423e-44b8-b54f-eaebde2bac7e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:01.010174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d7dc0-f907-4b32-8ece-bb6d7d9eb1c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:01.011885 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d7dc0-f907-4b32-8ece-bb6d7d9eb1c5 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:01.022189 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 323/647] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Aug 8 08:24:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:01.022865 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f45af7-423e-44b8-b54f-eaebde2bac7e None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:01.029351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-c31d7dc0-f907-4b32-8ece-bb6d7d9eb1c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:01.029351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:01.029351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:01.029351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:01.029351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:01.029351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:01.029351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:01.029351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:01.029351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:01.029351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:01.029351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:01.029351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:01.029351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:01.029351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:01.029351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:01.029351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:01.031183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:01.031183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:01.031183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:01.031183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:01.031183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:01.031183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:01.031183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:01.031183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:01.031183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:01.031183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:01.031183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:01.031183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:01.031183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:01.031183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:01.031183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:01.031183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:01.031183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:01.032271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:01.032271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:01.032271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:01.038126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 325/648] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:01.038126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d7dc0-f907-4b32-8ece-bb6d7d9eb1c5 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:01.099945 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:01.130667 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:01.137978 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f45af7-423e-44b8-b54f-eaebde2bac7e None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:01.140972 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d7dc0-f907-4b32-8ece-bb6d7d9eb1c5 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:01.174570 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34004cbe-9b95-457f-a914-2961579b740a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:01.175171 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34004cbe-9b95-457f-a914-2961579b740a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:01.175448 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34004cbe-9b95-457f-a914-2961579b740a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:01.175721 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34004cbe-9b95-457f-a914-2961579b740a None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:01.176314 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74052529-b540-4af6-b88f-b74e5cf44de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:01.176421 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34004cbe-9b95-457f-a914-2961579b740a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:01.176690 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34004cbe-9b95-457f-a914-2961579b740a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:01.177204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74052529-b540-4af6-b88f-b74e5cf44de8 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:01.177587 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74052529-b540-4af6-b88f-b74e5cf44de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:01.177966 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74052529-b540-4af6-b88f-b74e5cf44de8 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:01.178126 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34004cbe-9b95-457f-a914-2961579b740a None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:01.178852 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74052529-b540-4af6-b88f-b74e5cf44de8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:01.179260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74052529-b540-4af6-b88f-b74e5cf44de8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:01.180706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74052529-b540-4af6-b88f-b74e5cf44de8 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:01.197178 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 326/649] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 08:24:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:01.197178 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74052529-b540-4af6-b88f-b74e5cf44de8 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:01.197415 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 324/650] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:24:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:01.197415 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34004cbe-9b95-457f-a914-2961579b740a None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:01.303657 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34004cbe-9b95-457f-a914-2961579b740a None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:01.304021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74052529-b540-4af6-b88f-b74e5cf44de8 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:01.329012 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:01.338430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120e2f5-fcde-4afb-9de0-4de465213de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:01.339139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120e2f5-fcde-4afb-9de0-4de465213de2 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:01.339535 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120e2f5-fcde-4afb-9de0-4de465213de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:01.340182 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120e2f5-fcde-4afb-9de0-4de465213de2 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:01.340282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9fcfc-c5b7-493a-8942-0651650fbdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:01.340495 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9fcfc-c5b7-493a-8942-0651650fbdd4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:01.340709 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d120e2f5-fcde-4afb-9de0-4de465213de2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:01.340861 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9fcfc-c5b7-493a-8942-0651650fbdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:01.341116 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d120e2f5-fcde-4afb-9de0-4de465213de2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:01.341236 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9fcfc-c5b7-493a-8942-0651650fbdd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:01.342657 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d120e2f5-fcde-4afb-9de0-4de465213de2 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:01.360814 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 325/651] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:01.360814 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120e2f5-fcde-4afb-9de0-4de465213de2 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:01.375118 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:01.420584 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9fcfc-c5b7-493a-8942-0651650fbdd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:01.422856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9fcfc-c5b7-493a-8942-0651650fbdd4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:01.468751 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120e2f5-fcde-4afb-9de0-4de465213de2 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:01.511466 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e786b5f-2404-49e5-995a-77f311312af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:01.512582 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e786b5f-2404-49e5-995a-77f311312af9 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:01.512868 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e786b5f-2404-49e5-995a-77f311312af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:01.513154 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e786b5f-2404-49e5-995a-77f311312af9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:01.514154 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e786b5f-2404-49e5-995a-77f311312af9 None admin] RBAC: Adding query filter params (role.id=402c3e6fe3364285bd72784a297db11e, scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4, user.id=a3317f751ec74bde95ec262394001292) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:01.514489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e786b5f-2404-49e5-995a-77f311312af9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:01.516847 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e786b5f-2404-49e5-995a-77f311312af9 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:01.532236 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 326/652] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:24:01 2022] GET /identity/v3/role_assignments?user.id=a3317f751ec74bde95ec262394001292&scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4&role.id=402c3e6fe3364285bd72784a297db11e => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:01.532236 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e786b5f-2404-49e5-995a-77f311312af9 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:01.532542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 327/653] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:24:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:24:01.532542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9fcfc-c5b7-493a-8942-0651650fbdd4 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:01.561768 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:01.641593 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:01.652899 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9fcfc-c5b7-493a-8942-0651650fbdd4 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:01.665222 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e786b5f-2404-49e5-995a-77f311312af9 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:01.696720 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d6204-fbcf-48e4-a5de-e4c4077bbe88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:01.697911 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d6204-fbcf-48e4-a5de-e4c4077bbe88 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:01.697911 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d6204-fbcf-48e4-a5de-e4c4077bbe88 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:01.697911 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d6204-fbcf-48e4-a5de-e4c4077bbe88 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:01.706853 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4d6204-fbcf-48e4-a5de-e4c4077bbe88 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:01.707272 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4d6204-fbcf-48e4-a5de-e4c4077bbe88 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:01.708674 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474d824-8744-4fdf-af2b-caf14262eca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:01.709336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474d824-8744-4fdf-af2b-caf14262eca6 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:01.709798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474d824-8744-4fdf-af2b-caf14262eca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:01.710093 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474d824-8744-4fdf-af2b-caf14262eca6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:01.721060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-7e4d6204-fbcf-48e4-a5de-e4c4077bbe88 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 08:24:01.721060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:01.721060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:01.721060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:01.721060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:01.721060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:01.721060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:01.721060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:01.721060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:01.721060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:01.721060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:01.721060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:01.721060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:24:01.721060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:24:01.721060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:24:01.721060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:24:01.722137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:01.722137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:01.722137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:24:01.722137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:01.722137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:24:01.722137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:01.722137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:01.722137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:01.722137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:01.722137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:01.722137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:01.722137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:01.722137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:01.722137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:01.722137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:01.722137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:01.722137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:01.722137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:01.723147 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:01.723147 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:01.723147 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:24:01.723147 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:24:01.723147 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:24:01.723147 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:24:01.723147 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:24:01.723147 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:24:01.723147 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 08:24:01.723147 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:01.723692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6474d824-8744-4fdf-af2b-caf14262eca6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:01.723764 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 328/654] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 08:24:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:01.724288 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d6204-fbcf-48e4-a5de-e4c4077bbe88 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:01.725082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6474d824-8744-4fdf-af2b-caf14262eca6 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:01.747633 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-6474d824-8744-4fdf-af2b-caf14262eca6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:01.747633 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:01.747633 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:01.747633 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:01.747633 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:01.747633 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:01.747633 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:01.747633 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:01.747633 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:01.747633 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:01.747633 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:01.747633 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:01.747633 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:01.747633 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:01.747633 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:01.747633 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:01.748824 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:01.748824 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:01.748824 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:01.748824 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:01.748824 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:01.748824 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:01.748824 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:01.748824 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:01.748824 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:01.748824 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:01.748824 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:01.748824 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:01.748824 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:01.748824 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:01.748824 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:01.748824 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:01.748824 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:01.749990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:01.749990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:01.749990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:01.749990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 327/655] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:01.750240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474d824-8744-4fdf-af2b-caf14262eca6 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:01.849275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:01.869768 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d6204-fbcf-48e4-a5de-e4c4077bbe88 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:01.876505 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474d824-8744-4fdf-af2b-caf14262eca6 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:01.906530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:01.912831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545c069-1939-4f72-b238-6a6ddb6d1907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:01.913579 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56025ea0-710d-4a4a-b08f-47ef89e025e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:01.913873 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545c069-1939-4f72-b238-6a6ddb6d1907 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:01.914211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56025ea0-710d-4a4a-b08f-47ef89e025e9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:01.914450 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545c069-1939-4f72-b238-6a6ddb6d1907 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:01.914561 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56025ea0-710d-4a4a-b08f-47ef89e025e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:01.914745 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56025ea0-710d-4a4a-b08f-47ef89e025e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:01.915106 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545c069-1939-4f72-b238-6a6ddb6d1907 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:01.915978 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545c069-1939-4f72-b238-6a6ddb6d1907 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:01.916423 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545c069-1939-4f72-b238-6a6ddb6d1907 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:01.918011 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545c069-1939-4f72-b238-6a6ddb6d1907 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:01.927818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56025ea0-710d-4a4a-b08f-47ef89e025e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:01.930369 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56025ea0-710d-4a4a-b08f-47ef89e025e9 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:01.937223 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 328/656] 10.222.0.33 () {56 vars in 1139 bytes} [Mon Aug 8 08:24:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:01.938246 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545c069-1939-4f72-b238-6a6ddb6d1907 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:01.953635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-56025ea0-710d-4a4a-b08f-47ef89e025e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:01.953635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:01.953635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:01.953635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:01.953635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:01.953635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:01.953635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:01.953635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:01.953635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:01.953635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:01.953635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:01.953635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:01.953635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:01.953635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:01.953635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:01.953635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:01.954760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:01.954760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:01.954760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:01.954760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:01.954760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:01.954760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:01.954760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:01.954760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:01.954760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:01.954760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:01.954760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:01.954760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:01.954760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:01.954760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:01.954760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:01.954760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:01.954760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:01.955714 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:01.955714 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:01.955714 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:01.955714 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 329/657] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:01.960408 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56025ea0-710d-4a4a-b08f-47ef89e025e9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:02.050104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545c069-1939-4f72-b238-6a6ddb6d1907 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:02.062140 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56025ea0-710d-4a4a-b08f-47ef89e025e9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:02.084332 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be9c7d-14e5-4661-ba45-76535973d239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:02.084939 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be9c7d-14e5-4661-ba45-76535973d239 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:02.085249 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be9c7d-14e5-4661-ba45-76535973d239 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:02.085518 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be9c7d-14e5-4661-ba45-76535973d239 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:02.086151 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be9c7d-14e5-4661-ba45-76535973d239 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:02.086453 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be9c7d-14e5-4661-ba45-76535973d239 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:02.087803 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be9c7d-14e5-4661-ba45-76535973d239 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:02.100383 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 329/658] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:02.101277 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be9c7d-14e5-4661-ba45-76535973d239 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:02.105523 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0cc12c-85e6-4d1d-8e87-1ba82f1883ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:02.106143 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0cc12c-85e6-4d1d-8e87-1ba82f1883ea None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:02.106425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0cc12c-85e6-4d1d-8e87-1ba82f1883ea None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:02.106694 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0cc12c-85e6-4d1d-8e87-1ba82f1883ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:02.111599 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:02.159317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:02.176118 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0cc12c-85e6-4d1d-8e87-1ba82f1883ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:02.177583 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0cc12c-85e6-4d1d-8e87-1ba82f1883ea None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:02.216387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be9c7d-14e5-4661-ba45-76535973d239 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:02.256366 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4e9a4-97c5-4b9a-b132-e0a1dc51d8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:02.256971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4e9a4-97c5-4b9a-b132-e0a1dc51d8ab None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:02.257351 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4e9a4-97c5-4b9a-b132-e0a1dc51d8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:02.257623 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4e9a4-97c5-4b9a-b132-e0a1dc51d8ab None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:02.258256 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f4e9a4-97c5-4b9a-b132-e0a1dc51d8ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:02.258634 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f4e9a4-97c5-4b9a-b132-e0a1dc51d8ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:02.260109 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f4e9a4-97c5-4b9a-b132-e0a1dc51d8ab None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:02.276648 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 330/659] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:02.276648 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a5f65-2d29-4eb6-a23c-95084ebe22b1 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:02.276648 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a5f65-2d29-4eb6-a23c-95084ebe22b1 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:02.276648 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a5f65-2d29-4eb6-a23c-95084ebe22b1 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:02.276648 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 331/660] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:02.276648 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a5f65-2d29-4eb6-a23c-95084ebe22b1 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:02.293540 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 330/661] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:24:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:24:02.294454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0cc12c-85e6-4d1d-8e87-1ba82f1883ea None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:02.333849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:02.388710 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a5f65-2d29-4eb6-a23c-95084ebe22b1 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:02.398967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:02.401316 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0cc12c-85e6-4d1d-8e87-1ba82f1883ea None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:02.428767 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a68829-fb1c-48c2-86b5-513bbe838038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:02.429428 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a68829-fb1c-48c2-86b5-513bbe838038 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:02.429709 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a68829-fb1c-48c2-86b5-513bbe838038 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:02.430033 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a68829-fb1c-48c2-86b5-513bbe838038 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:02.437515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a68829-fb1c-48c2-86b5-513bbe838038 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:02.438940 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a68829-fb1c-48c2-86b5-513bbe838038 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:02.441681 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2b675-a0b6-42c1-8027-0caa253f1b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:02.442307 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2b675-a0b6-42c1-8027-0caa253f1b0d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:02.442581 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2b675-a0b6-42c1-8027-0caa253f1b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:02.442849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2b675-a0b6-42c1-8027-0caa253f1b0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:02.449461 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e2b675-a0b6-42c1-8027-0caa253f1b0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:02.450894 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e2b675-a0b6-42c1-8027-0caa253f1b0d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:02.451975 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-a5a68829-fb1c-48c2-86b5-513bbe838038 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 08:24:02.451975 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:02.451975 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:02.451975 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:02.451975 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:02.451975 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:02.451975 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:02.451975 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:02.451975 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:02.451975 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:02.451975 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:02.451975 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:02.451975 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:24:02.451975 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:24:02.451975 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:24:02.451975 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:24:02.453104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:02.453104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:02.453104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:24:02.453104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:02.453104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:24:02.453104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:02.453104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:02.453104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:02.453104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:02.453104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:02.453104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:02.453104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:02.453104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:02.453104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:02.453104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:02.453104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:02.453104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:02.453104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:02.454111 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:02.454111 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:02.454111 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:24:02.454111 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:24:02.454111 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:24:02.454111 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:24:02.454111 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:24:02.454111 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:24:02.454111 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 08:24:02.454111 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:02.455915 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 332/662] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:24:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:02.457466 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf9abe-7796-4ab5-ba19-f05872cd4a3f None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:02.457744 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf9abe-7796-4ab5-ba19-f05872cd4a3f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:02.458014 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf9abe-7796-4ab5-ba19-f05872cd4a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:02.468444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-72e2b675-a0b6-42c1-8027-0caa253f1b0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:02.468444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:02.468444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:02.468444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:02.468444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:02.468444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:02.468444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:02.468444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:02.468444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:02.468444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:02.468444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:02.468444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:02.468444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:24:02.468444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:24:02.468444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:24:02.475135 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:24:02.475135 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:02.475135 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:02.475135 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:02.475135 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:02.475135 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:02.475135 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:02.475135 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:02.475135 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:02.475135 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:02.475135 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:02.475135 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:02.475135 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:02.475135 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:02.475135 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:02.475135 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:02.475135 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:02.476074 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:24:02.476074 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:24:02.476074 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:24:02.476074 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:24:02.476074 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:24:02.476074 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:24:02.476074 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:02.476074 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:02.476074 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 331/663] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:24:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:02.476074 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2b675-a0b6-42c1-8027-0caa253f1b0d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:02.478533 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-5dbf9abe-7796-4ab5-ba19-f05872cd4a3f None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:02.479974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5dbf9abe-7796-4ab5-ba19-f05872cd4a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:02.552673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 333/664] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:24:02.553877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf9abe-7796-4ab5-ba19-f05872cd4a3f None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:02.582322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:02.588384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2b675-a0b6-42c1-8027-0caa253f1b0d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:02.628749 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39053923-b995-46f8-ac38-54570c1cf224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:02.629420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39053923-b995-46f8-ac38-54570c1cf224 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:02.629698 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39053923-b995-46f8-ac38-54570c1cf224 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:02.629966 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39053923-b995-46f8-ac38-54570c1cf224 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:02.640904 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39053923-b995-46f8-ac38-54570c1cf224 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:02.642810 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39053923-b995-46f8-ac38-54570c1cf224 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:02.653695 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:02.658109 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf9abe-7796-4ab5-ba19-f05872cd4a3f None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:02.658443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-39053923-b995-46f8-ac38-54570c1cf224 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:02.658443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:02.658443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:02.658443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:02.658443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:02.658443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:02.658443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:02.658443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:02.658443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:02.658443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:02.658443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:02.658443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:02.658443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:02.658443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:02.658443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:02.658443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:02.659405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:02.659405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:02.659405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:02.659405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:02.659405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:02.659405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:02.659405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:02.659405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:02.659405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:02.659405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:02.659405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:02.659405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:02.659405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:02.659405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:02.659405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:02.659405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:02.659405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:02.660298 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:02.660298 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:02.660298 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:02.660298 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 332/665] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:02.660298 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39053923-b995-46f8-ac38-54570c1cf224 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:02.698732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5dc0cc-cb2b-4f37-acc4-a36d15a14e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:02.699337 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5dc0cc-cb2b-4f37-acc4-a36d15a14e5e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:02.699622 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5dc0cc-cb2b-4f37-acc4-a36d15a14e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:02.699896 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5dc0cc-cb2b-4f37-acc4-a36d15a14e5e None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:02.700635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5dc0cc-cb2b-4f37-acc4-a36d15a14e5e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:02.700918 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5dc0cc-cb2b-4f37-acc4-a36d15a14e5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:02.703145 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5dc0cc-cb2b-4f37-acc4-a36d15a14e5e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:02.721358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 334/666] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Aug 8 08:24:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:02.722096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5dc0cc-cb2b-4f37-acc4-a36d15a14e5e None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:02.772624 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39053923-b995-46f8-ac38-54570c1cf224 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:02.810529 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ad22f-a5c4-488c-9085-c3eaf9676469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:02.811284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ad22f-a5c4-488c-9085-c3eaf9676469 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:02.812253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ad22f-a5c4-488c-9085-c3eaf9676469 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:02.812561 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ad22f-a5c4-488c-9085-c3eaf9676469 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:02.813382 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ad22f-a5c4-488c-9085-c3eaf9676469 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:02.813690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ad22f-a5c4-488c-9085-c3eaf9676469 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:02.815250 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ad22f-a5c4-488c-9085-c3eaf9676469 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:02.826717 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5dc0cc-cb2b-4f37-acc4-a36d15a14e5e None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:02.832856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 333/667] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:24:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:02.833760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ad22f-a5c4-488c-9085-c3eaf9676469 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:02.844614 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:02.863910 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426db53-11fd-4447-a67d-f9e281601516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:02.864542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426db53-11fd-4447-a67d-f9e281601516 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:02.864818 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426db53-11fd-4447-a67d-f9e281601516 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:02.865091 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426db53-11fd-4447-a67d-f9e281601516 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:02.875483 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c426db53-11fd-4447-a67d-f9e281601516 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:24:02.880113 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-c426db53-11fd-4447-a67d-f9e281601516 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:24:02.880113 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:02.880113 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:02.880113 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:02.880113 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:02.880113 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:02.880113 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:02.880113 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:02.880113 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:02.880113 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:02.880113 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:02.880113 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:02.880113 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:24:02.880113 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:24:02.880113 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:24:02.880113 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:24:02.886541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:24:02.886541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:24:02.886541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:24:02.886541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:24:02.886541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:02.886541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:02.886541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:02.886541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:02.886541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:02.886541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:02.886541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:02.886541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:02.886541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:02.886541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:02.886541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:02.886541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:02.887550 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:02.887550 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:02.887550 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:02.887550 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:02.887550 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:24:02.887550 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:24:02.887550 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:24:02.887550 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:24:02.887550 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:24:02.887550 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:24:02.887550 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:24:02.887550 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:02.887550 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 335/668] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 08:24:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:02.887550 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426db53-11fd-4447-a67d-f9e281601516 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:02.905655 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:02.947773 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ad22f-a5c4-488c-9085-c3eaf9676469 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:02.986628 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccdaefb-5372-4494-a3cb-29cea564f581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:02.987259 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdaefb-5372-4494-a3cb-29cea564f581 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:02.987572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdaefb-5372-4494-a3cb-29cea564f581 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:02.987908 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdaefb-5372-4494-a3cb-29cea564f581 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:02.988550 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccdaefb-5372-4494-a3cb-29cea564f581 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:02.988850 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccdaefb-5372-4494-a3cb-29cea564f581 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:02.989545 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426db53-11fd-4447-a67d-f9e281601516 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:02.990384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccdaefb-5372-4494-a3cb-29cea564f581 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:03.007774 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 334/669] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:03.008409 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccdaefb-5372-4494-a3cb-29cea564f581 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:03.029738 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b59d44-22df-4566-8e03-65d07458bcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:03.031390 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b59d44-22df-4566-8e03-65d07458bcc1 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:03.031676 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b59d44-22df-4566-8e03-65d07458bcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:03.031944 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b59d44-22df-4566-8e03-65d07458bcc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:03.079465 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:03.105169 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b59d44-22df-4566-8e03-65d07458bcc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:03.106726 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b59d44-22df-4566-8e03-65d07458bcc1 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:03.112660 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccdaefb-5372-4494-a3cb-29cea564f581 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:03.124791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:03.150913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e542e46-fc65-4a70-b7e8-321b0d241692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:03.151518 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e542e46-fc65-4a70-b7e8-321b0d241692 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:03.151854 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e542e46-fc65-4a70-b7e8-321b0d241692 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:03.152129 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e542e46-fc65-4a70-b7e8-321b0d241692 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:03.152852 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e542e46-fc65-4a70-b7e8-321b0d241692 None admin] RBAC: Adding query filter params (role.id=402c3e6fe3364285bd72784a297db11e, scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4, user.id=627171bf2a2e4a21ba6f800879f8c665) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:03.153171 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e542e46-fc65-4a70-b7e8-321b0d241692 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:03.154616 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e542e46-fc65-4a70-b7e8-321b0d241692 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:03.160612 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 335/670] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:24:03 2022] GET /identity/v3/role_assignments?user.id=627171bf2a2e4a21ba6f800879f8c665&scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4&role.id=402c3e6fe3364285bd72784a297db11e => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:03.161243 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e542e46-fc65-4a70-b7e8-321b0d241692 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:03.247422 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 336/671] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:24:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:24:03.248692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b59d44-22df-4566-8e03-65d07458bcc1 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:03.290205 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e542e46-fc65-4a70-b7e8-321b0d241692 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:03.323510 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:03.329461 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2b6d08-1d30-411b-a624-b223e4dbf04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:03.330137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b6d08-1d30-411b-a624-b223e4dbf04b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:03.330421 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b6d08-1d30-411b-a624-b223e4dbf04b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:03.330691 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b6d08-1d30-411b-a624-b223e4dbf04b None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:03.331355 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2b6d08-1d30-411b-a624-b223e4dbf04b None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:03.331658 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2b6d08-1d30-411b-a624-b223e4dbf04b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:03.333310 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2b6d08-1d30-411b-a624-b223e4dbf04b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:03.343666 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 336/672] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 08:24:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:03.344401 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2b6d08-1d30-411b-a624-b223e4dbf04b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:03.386467 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b59d44-22df-4566-8e03-65d07458bcc1 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:03.404979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:03.433695 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c52940-317e-4448-89cf-5e9e158835d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:03.435026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c52940-317e-4448-89cf-5e9e158835d9 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:03.435026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c52940-317e-4448-89cf-5e9e158835d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:03.435026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c52940-317e-4448-89cf-5e9e158835d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:03.446507 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c52940-317e-4448-89cf-5e9e158835d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:03.448180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c52940-317e-4448-89cf-5e9e158835d9 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:03.462268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-92c52940-317e-4448-89cf-5e9e158835d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:03.462268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:03.462268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:03.462268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:03.462268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:03.462268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:03.462268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:03.462268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:03.462268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:03.462268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:03.462268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:03.462268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:03.462268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:24:03.462268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:24:03.462268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:24:03.463355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:24:03.463355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:03.463355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:03.463355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:03.463355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:03.463355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:03.463355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:03.463355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:03.463355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:03.463355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:03.463355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:03.463355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:03.463355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:03.463355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:03.463355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:03.463355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:03.463355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:03.464432 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:24:03.464432 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:24:03.464432 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:24:03.464432 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:24:03.464432 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:24:03.464432 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:24:03.464432 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:03.464432 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:03.464432 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 337/673] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:24:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:03.464912 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c52940-317e-4448-89cf-5e9e158835d9 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:03.485476 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2b6d08-1d30-411b-a624-b223e4dbf04b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:03.535509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457a2c7-8039-4df7-b308-94e9d2b9b688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:03.536375 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457a2c7-8039-4df7-b308-94e9d2b9b688 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:03.536792 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457a2c7-8039-4df7-b308-94e9d2b9b688 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:03.537466 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457a2c7-8039-4df7-b308-94e9d2b9b688 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:03.552908 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457a2c7-8039-4df7-b308-94e9d2b9b688 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:03.554893 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457a2c7-8039-4df7-b308-94e9d2b9b688 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:03.574429 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-e457a2c7-8039-4df7-b308-94e9d2b9b688 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:03.574429 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:03.574429 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:03.574429 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:03.574429 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:03.574429 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:03.574429 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:03.574429 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:03.574429 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:03.574429 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:03.574429 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:03.574429 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:03.574429 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:03.574429 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:03.574429 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:03.574429 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:03.575643 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:03.575643 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:03.575643 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:03.575643 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:03.575643 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:03.575643 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:03.575643 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:03.575643 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:03.575643 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:03.575643 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:03.575643 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:03.575643 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:03.575643 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:03.575643 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:03.575643 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:03.575643 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:03.575643 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:03.576737 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:03.576737 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:03.576737 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:03.577517 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c52940-317e-4448-89cf-5e9e158835d9 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:03.580465 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 337/674] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:03.581465 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457a2c7-8039-4df7-b308-94e9d2b9b688 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:03.623572 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e3efa-aaf8-4b81-86bb-2cd35262cc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:03.624266 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e3efa-aaf8-4b81-86bb-2cd35262cc72 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:03.624542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e3efa-aaf8-4b81-86bb-2cd35262cc72 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:03.624808 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e3efa-aaf8-4b81-86bb-2cd35262cc72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:03.635588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e3efa-aaf8-4b81-86bb-2cd35262cc72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:03.637033 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e3efa-aaf8-4b81-86bb-2cd35262cc72 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:03.650115 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:03.654258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-dd8e3efa-aaf8-4b81-86bb-2cd35262cc72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:03.654258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:03.654258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:03.654258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:03.654258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:03.654258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:03.654258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:03.654258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:03.654258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:03.654258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:03.654258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:03.654258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:03.654258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:03.654258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:03.654258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:03.654258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:03.655347 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:03.655347 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:03.655347 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:03.655347 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:03.655347 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:03.655347 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:03.655347 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:03.655347 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:03.655347 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:03.655347 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:03.655347 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:03.655347 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:03.655347 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:03.655347 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:03.655347 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:03.655347 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:03.655347 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:03.656256 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:03.656256 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:03.656256 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:03.660442 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 338/675] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:03.661207 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e3efa-aaf8-4b81-86bb-2cd35262cc72 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:03.699489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:03.714805 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457a2c7-8039-4df7-b308-94e9d2b9b688 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:03.751508 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d4dfa-a511-4f8c-98b6-d719f6093a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:03.752120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d4dfa-a511-4f8c-98b6-d719f6093a94 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:03.752398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d4dfa-a511-4f8c-98b6-d719f6093a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:03.752669 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d4dfa-a511-4f8c-98b6-d719f6093a94 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:03.753382 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d4dfa-a511-4f8c-98b6-d719f6093a94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:03.753685 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d4dfa-a511-4f8c-98b6-d719f6093a94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:03.755103 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d4dfa-a511-4f8c-98b6-d719f6093a94 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:03.768863 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 338/676] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:24:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:03.769839 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d4dfa-a511-4f8c-98b6-d719f6093a94 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:03.786284 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e3efa-aaf8-4b81-86bb-2cd35262cc72 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:03.821461 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efeb705-b72f-44b1-a7c7-cfb622629a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:03.822064 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efeb705-b72f-44b1-a7c7-cfb622629a17 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:03.822340 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efeb705-b72f-44b1-a7c7-cfb622629a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:03.822610 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efeb705-b72f-44b1-a7c7-cfb622629a17 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:03.823246 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efeb705-b72f-44b1-a7c7-cfb622629a17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:03.823545 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efeb705-b72f-44b1-a7c7-cfb622629a17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:03.824895 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efeb705-b72f-44b1-a7c7-cfb622629a17 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:03.837617 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 339/677] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:03.838798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cd0a9-e49e-40ed-aed8-3edb2d632b86 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:03.839084 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cd0a9-e49e-40ed-aed8-3edb2d632b86 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:03.839480 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cd0a9-e49e-40ed-aed8-3edb2d632b86 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:03.845176 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 340/678] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:03.845176 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cd0a9-e49e-40ed-aed8-3edb2d632b86 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:03.886360 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d4dfa-a511-4f8c-98b6-d719f6093a94 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:03.894270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:03.922748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bedffc-3b07-4632-8c42-08d2c64964a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:03.923365 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bedffc-3b07-4632-8c42-08d2c64964a4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:03.924064 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bedffc-3b07-4632-8c42-08d2c64964a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:03.924725 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bedffc-3b07-4632-8c42-08d2c64964a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:03.925634 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bedffc-3b07-4632-8c42-08d2c64964a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:03.926292 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bedffc-3b07-4632-8c42-08d2c64964a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:03.928367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bedffc-3b07-4632-8c42-08d2c64964a4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:03.938879 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:03.944776 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 339/679] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:03.944776 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bedffc-3b07-4632-8c42-08d2c64964a4 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:03.957928 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cd0a9-e49e-40ed-aed8-3edb2d632b86 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:03.993781 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f96ed9-e095-49ac-9686-6aa8843da1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:03.994389 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f96ed9-e095-49ac-9686-6aa8843da1d2 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:03.994673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f96ed9-e095-49ac-9686-6aa8843da1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:03.994943 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f96ed9-e095-49ac-9686-6aa8843da1d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:03.995659 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f96ed9-e095-49ac-9686-6aa8843da1d2 None admin] RBAC: Adding query filter params (role.id=402c3e6fe3364285bd72784a297db11e, scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4, user.id=2c3b8b43026c40209bfc42acd2830610) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:03.995973 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f96ed9-e095-49ac-9686-6aa8843da1d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:03.997395 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f96ed9-e095-49ac-9686-6aa8843da1d2 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:04.004126 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 341/680] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:24:03 2022] GET /identity/v3/role_assignments?user.id=2c3b8b43026c40209bfc42acd2830610&scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4&role.id=402c3e6fe3364285bd72784a297db11e => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:04.005314 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca311206-839d-41a8-b673-036091aa4aeb None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:04.005608 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca311206-839d-41a8-b673-036091aa4aeb None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:04.005882 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca311206-839d-41a8-b673-036091aa4aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:04.025456 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-ca311206-839d-41a8-b673-036091aa4aeb None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:04.026669 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ca311206-839d-41a8-b673-036091aa4aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:04.078699 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bedffc-3b07-4632-8c42-08d2c64964a4 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:04.120718 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 342/681] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:24:04.120718 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca311206-839d-41a8-b673-036091aa4aeb None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:04.123523 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15dbb5-2d53-4791-ba98-b00305c09108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:04.125204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15dbb5-2d53-4791-ba98-b00305c09108 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:04.125204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15dbb5-2d53-4791-ba98-b00305c09108 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:04.125204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15dbb5-2d53-4791-ba98-b00305c09108 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:04.138561 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15dbb5-2d53-4791-ba98-b00305c09108 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:04.140607 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15dbb5-2d53-4791-ba98-b00305c09108 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:04.160138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-eb15dbb5-2d53-4791-ba98-b00305c09108 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:04.160138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:04.160138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:04.160138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:04.160138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:04.160138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:04.160138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:04.160138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:04.160138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:04.160138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:04.160138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:04.160138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:04.160138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:24:04.160138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:24:04.160138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:24:04.161082 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:24:04.161082 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:04.161082 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:04.161082 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:04.161082 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:04.161082 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:04.161082 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:04.161082 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:04.161082 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:04.161082 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:04.161082 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:04.161082 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:04.161082 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:04.161082 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:04.161082 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:04.161082 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:04.161082 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:04.162037 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:24:04.162037 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:24:04.162037 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:24:04.162037 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:24:04.162037 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:24:04.162037 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:24:04.162037 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:04.162037 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:04.164446 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 340/682] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:24:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:04.165464 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15dbb5-2d53-4791-ba98-b00305c09108 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:04.198477 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:04.241466 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:04.265430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca311206-839d-41a8-b673-036091aa4aeb None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:04.281158 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15dbb5-2d53-4791-ba98-b00305c09108 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:04.304798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd2f22-5c1a-4654-8236-83cf7e37d9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:04.305679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd2f22-5c1a-4654-8236-83cf7e37d9c4 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:04.306157 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd2f22-5c1a-4654-8236-83cf7e37d9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:04.306573 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd2f22-5c1a-4654-8236-83cf7e37d9c4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:04.314471 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccd2f22-5c1a-4654-8236-83cf7e37d9c4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:04.316435 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccd2f22-5c1a-4654-8236-83cf7e37d9c4 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:04.318300 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76887136-7af2-4197-a50b-3c7ed0b91554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:04.319218 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76887136-7af2-4197-a50b-3c7ed0b91554 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:04.319788 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76887136-7af2-4197-a50b-3c7ed0b91554 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:04.320089 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76887136-7af2-4197-a50b-3c7ed0b91554 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:04.331248 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-fccd2f22-5c1a-4654-8236-83cf7e37d9c4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 08:24:04.331248 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:04.331248 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:04.331248 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:04.331248 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:04.331248 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:04.331248 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:04.331248 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:04.331248 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:04.331248 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:04.331248 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:04.331248 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:04.331248 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:24:04.331248 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:24:04.331248 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:24:04.331248 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:24:04.332446 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:04.332446 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:04.332446 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:24:04.332446 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:04.332446 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:24:04.332446 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:04.332446 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:04.332446 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:04.332446 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:04.332446 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:04.332446 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:04.332446 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:04.332446 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:04.332446 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:04.332446 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:04.332446 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:04.332446 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:04.332446 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:04.333414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:04.333414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:04.333414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:24:04.333414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:24:04.333414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:24:04.333414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:24:04.333414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:24:04.333414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:24:04.333414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 08:24:04.333414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:04.333977 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76887136-7af2-4197-a50b-3c7ed0b91554 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:24:04.340017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-76887136-7af2-4197-a50b-3c7ed0b91554 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:24:04.340017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:04.340017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:04.340017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:04.340017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:04.340017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:04.340017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:04.340017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:04.340017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:04.340017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:04.340017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:04.340017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:04.340017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:24:04.340017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:24:04.340017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:24:04.340017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:24:04.340961 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:24:04.340961 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:24:04.340961 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:24:04.340961 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:24:04.340961 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:04.340961 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:04.340961 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:04.340961 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:04.340961 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:04.340961 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:04.340961 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:04.340961 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:04.340961 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:04.340961 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:04.340961 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:04.340961 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:04.341848 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:04.341848 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:04.341848 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:04.341848 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:04.341848 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:24:04.341848 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:24:04.341848 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:24:04.341848 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:24:04.341848 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:24:04.341848 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:24:04.341848 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:24:04.341848 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:04.341848 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 341/684] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:24:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:04.341848 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76887136-7af2-4197-a50b-3c7ed0b91554 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:04.343574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 343/684] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:24:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:04.350254 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd2f22-5c1a-4654-8236-83cf7e37d9c4 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:04.445348 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:04.461592 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76887136-7af2-4197-a50b-3c7ed0b91554 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:04.486470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd2f22-5c1a-4654-8236-83cf7e37d9c4 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:04.507002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d3b39-2f73-4283-ae91-c87f44aedf3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:04.508284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d3b39-2f73-4283-ae91-c87f44aedf3e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:04.508711 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d3b39-2f73-4283-ae91-c87f44aedf3e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:04.509079 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d3b39-2f73-4283-ae91-c87f44aedf3e None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:04.510177 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d3b39-2f73-4283-ae91-c87f44aedf3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:04.510578 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d3b39-2f73-4283-ae91-c87f44aedf3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:04.513082 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d3b39-2f73-4283-ae91-c87f44aedf3e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:04.522989 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5fad4f-0959-4ae0-9e86-ab68505584f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:04.523901 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fad4f-0959-4ae0-9e86-ab68505584f1 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:04.524392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fad4f-0959-4ae0-9e86-ab68505584f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:04.524867 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fad4f-0959-4ae0-9e86-ab68505584f1 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:04.525817 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5fad4f-0959-4ae0-9e86-ab68505584f1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:04.526334 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5fad4f-0959-4ae0-9e86-ab68505584f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:04.528004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5fad4f-0959-4ae0-9e86-ab68505584f1 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:04.528111 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 342/685] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:24:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:04.528965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d3b39-2f73-4283-ae91-c87f44aedf3e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:04.545102 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:04.549930 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 344/686] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Aug 8 08:24:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:04.550989 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5fad4f-0959-4ae0-9e86-ab68505584f1 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:04.661888 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5fad4f-0959-4ae0-9e86-ab68505584f1 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:04.669273 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d3b39-2f73-4283-ae91-c87f44aedf3e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:04.698057 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e80c6-7143-4f10-ac7d-61b13a577026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:04.699060 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e80c6-7143-4f10-ac7d-61b13a577026 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:04.699534 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e80c6-7143-4f10-ac7d-61b13a577026 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:04.699996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e80c6-7143-4f10-ac7d-61b13a577026 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:04.700908 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245e80c6-7143-4f10-ac7d-61b13a577026 None admin] RBAC: Adding query filter params (role.id=402c3e6fe3364285bd72784a297db11e, scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4, user.id=3262fa2d9c0d4d8998c9697994b48fef) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:04.701434 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245e80c6-7143-4f10-ac7d-61b13a577026 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:04.703320 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245e80c6-7143-4f10-ac7d-61b13a577026 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:04.705095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0e24e-d01e-40b5-81f8-15506570333a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:04.705706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0e24e-d01e-40b5-81f8-15506570333a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:04.705975 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0e24e-d01e-40b5-81f8-15506570333a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:04.706243 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0e24e-d01e-40b5-81f8-15506570333a None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:04.706896 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0e24e-d01e-40b5-81f8-15506570333a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:04.707448 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0e24e-d01e-40b5-81f8-15506570333a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:04.708795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0e24e-d01e-40b5-81f8-15506570333a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:04.711297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 345/687] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:24:04 2022] GET /identity/v3/role_assignments?user.id=3262fa2d9c0d4d8998c9697994b48fef&scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4&role.id=402c3e6fe3364285bd72784a297db11e => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:04.712352 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e80c6-7143-4f10-ac7d-61b13a577026 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:04.723281 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 343/688] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:24:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:04.724716 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0e24e-d01e-40b5-81f8-15506570333a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:04.736732 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:04.785468 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:04.829501 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e80c6-7143-4f10-ac7d-61b13a577026 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:04.835492 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0e24e-d01e-40b5-81f8-15506570333a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:04.867847 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239d2e0-12c2-44c9-b34a-3e3512c129ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:04.869005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239d2e0-12c2-44c9-b34a-3e3512c129ce None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:04.869541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239d2e0-12c2-44c9-b34a-3e3512c129ce None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:04.870009 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239d2e0-12c2-44c9-b34a-3e3512c129ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:04.872385 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde6ae53-c8ec-4af8-819d-546f6a20269f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:04.873042 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6ae53-c8ec-4af8-819d-546f6a20269f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:04.873350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6ae53-c8ec-4af8-819d-546f6a20269f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:04.873616 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6ae53-c8ec-4af8-819d-546f6a20269f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:04.885217 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde6ae53-c8ec-4af8-819d-546f6a20269f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:04.886716 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde6ae53-c8ec-4af8-819d-546f6a20269f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:04.902473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-bde6ae53-c8ec-4af8-819d-546f6a20269f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:04.902473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:04.902473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:04.902473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:04.902473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:04.902473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:04.902473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:04.902473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:04.902473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:04.902473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:04.902473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:04.902473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:04.902473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:04.902473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:04.902473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:04.902473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:04.903475 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:04.903475 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:04.903475 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:04.903475 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:04.903475 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:04.903475 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:04.903475 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:04.903475 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:04.903475 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:04.903475 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:04.903475 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:04.903475 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:04.903475 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:04.903475 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:04.903475 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:04.903475 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:04.903475 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:04.904365 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:04.904365 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:04.904365 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:04.908590 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 344/689] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:04.910959 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde6ae53-c8ec-4af8-819d-546f6a20269f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:04.980683 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1239d2e0-12c2-44c9-b34a-3e3512c129ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:04.982664 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1239d2e0-12c2-44c9-b34a-3e3512c129ce None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:05.010132 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:05.062346 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde6ae53-c8ec-4af8-819d-546f6a20269f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:05.064392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:05.107500 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e1fbd-1b08-464d-b93d-2fac13fb3141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:05.108389 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e1fbd-1b08-464d-b93d-2fac13fb3141 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:05.109056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e1fbd-1b08-464d-b93d-2fac13fb3141 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:05.109369 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e1fbd-1b08-464d-b93d-2fac13fb3141 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:05.110279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601e1fbd-1b08-464d-b93d-2fac13fb3141 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:05.110582 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601e1fbd-1b08-464d-b93d-2fac13fb3141 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:05.112872 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601e1fbd-1b08-464d-b93d-2fac13fb3141 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:05.120672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 346/690] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:24:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:24:05.121665 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239d2e0-12c2-44c9-b34a-3e3512c129ce None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:05.126606 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 345/691] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:05.132605 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e1fbd-1b08-464d-b93d-2fac13fb3141 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:05.231463 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239d2e0-12c2-44c9-b34a-3e3512c129ce None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:05.240716 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e1fbd-1b08-464d-b93d-2fac13fb3141 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:05.261821 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:05.268033 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd586e8-74d9-4bf4-897d-f0e6d6d668f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:05.268716 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd586e8-74d9-4bf4-897d-f0e6d6d668f6 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:05.268998 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd586e8-74d9-4bf4-897d-f0e6d6d668f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:05.269322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd586e8-74d9-4bf4-897d-f0e6d6d668f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:05.278171 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcdca2-9c1f-4a45-861f-b9e1fd8f4a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:05.279086 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcdca2-9c1f-4a45-861f-b9e1fd8f4a0e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:05.279555 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcdca2-9c1f-4a45-861f-b9e1fd8f4a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:05.280551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcdca2-9c1f-4a45-861f-b9e1fd8f4a0e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:05.288399 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfcdca2-9c1f-4a45-861f-b9e1fd8f4a0e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:05.288556 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd586e8-74d9-4bf4-897d-f0e6d6d668f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:05.290017 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd586e8-74d9-4bf4-897d-f0e6d6d668f6 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:05.290549 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfcdca2-9c1f-4a45-861f-b9e1fd8f4a0e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:05.303172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-dbfcdca2-9c1f-4a45-861f-b9e1fd8f4a0e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 08:24:05.303172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:05.303172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:05.303172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:05.303172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:05.303172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:05.303172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:05.303172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:05.303172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:05.303172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:05.303172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:05.303172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:05.303172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:24:05.303172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:24:05.303172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:24:05.303172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:24:05.304238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:05.304238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:05.304238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:24:05.304238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:05.304238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:24:05.304238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:05.304238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:05.304238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:05.304238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:05.304238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:05.304238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:05.304238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:05.304238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:05.304238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:05.304238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:05.304238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:05.304238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:05.304238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:05.305183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:05.305183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:05.305183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:24:05.305183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:24:05.305183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:24:05.305183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:24:05.305183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:24:05.305183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:24:05.305183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 08:24:05.305183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:05.312439 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 346/692] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Aug 8 08:24:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:24:05.312577 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-9bd586e8-74d9-4bf4-897d-f0e6d6d668f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:05.312577 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:05.312577 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:05.312577 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:05.312577 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:05.312577 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:05.312577 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:05.312577 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:05.312577 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:05.312577 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:05.312577 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:05.312577 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:05.312577 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:05.312577 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:05.312577 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:05.312577 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:05.313474 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:05.313474 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:05.313474 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:05.313474 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:05.313474 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:05.313474 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:05.313474 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:05.313474 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:05.313474 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:05.313474 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:05.313474 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:05.313474 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:05.313474 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:05.313474 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:05.313474 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:05.313474 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:05.313474 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:05.314343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:05.314343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:05.314343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:05.316171 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 347/693] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:05.316689 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcdca2-9c1f-4a45-861f-b9e1fd8f4a0e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:05.322240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd586e8-74d9-4bf4-897d-f0e6d6d668f6 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:05.353126 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:05.457414 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcdca2-9c1f-4a45-861f-b9e1fd8f4a0e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:05.477099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd586e8-74d9-4bf4-897d-f0e6d6d668f6 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:05.499668 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45d031-fd08-454d-8be6-4f2a211f481f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:05.500690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45d031-fd08-454d-8be6-4f2a211f481f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:05.501259 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45d031-fd08-454d-8be6-4f2a211f481f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:05.502297 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45d031-fd08-454d-8be6-4f2a211f481f None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:05.503519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45d031-fd08-454d-8be6-4f2a211f481f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:05.504027 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45d031-fd08-454d-8be6-4f2a211f481f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:05.506573 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45d031-fd08-454d-8be6-4f2a211f481f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:05.516125 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad36e08-9a13-48f9-9aa0-93ab51a74296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:05.516740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad36e08-9a13-48f9-9aa0-93ab51a74296 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:05.517018 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad36e08-9a13-48f9-9aa0-93ab51a74296 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:05.517315 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad36e08-9a13-48f9-9aa0-93ab51a74296 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:05.517954 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad36e08-9a13-48f9-9aa0-93ab51a74296 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:05.518251 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad36e08-9a13-48f9-9aa0-93ab51a74296 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:05.520464 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad36e08-9a13-48f9-9aa0-93ab51a74296 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:05.531180 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 347/694] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Aug 8 08:24:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:05.532788 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c99a5a-8204-4bbb-939a-36e218f733e9 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:05.533305 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c99a5a-8204-4bbb-939a-36e218f733e9 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:05.533783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c99a5a-8204-4bbb-939a-36e218f733e9 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:05.535631 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 348/695] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:05.539857 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c99a5a-8204-4bbb-939a-36e218f733e9 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:05.547635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 348/696] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:05.548712 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781fa1f-6f80-4eee-a1ad-353a713f67d0 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:05.548995 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781fa1f-6f80-4eee-a1ad-353a713f67d0 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:05.549297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781fa1f-6f80-4eee-a1ad-353a713f67d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:05.568387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-3781fa1f-6f80-4eee-a1ad-353a713f67d0 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:05.569493 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-3781fa1f-6f80-4eee-a1ad-353a713f67d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:05.600071 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:05.649167 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 349/697] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:24:05.649167 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781fa1f-6f80-4eee-a1ad-353a713f67d0 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:05.661900 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:05.704848 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c99a5a-8204-4bbb-939a-36e218f733e9 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:05.743118 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef97de8-38c8-4110-9487-39b8dbc2ebf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:05.744566 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef97de8-38c8-4110-9487-39b8dbc2ebf6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:05.745089 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef97de8-38c8-4110-9487-39b8dbc2ebf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:05.745682 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef97de8-38c8-4110-9487-39b8dbc2ebf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:05.767842 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781fa1f-6f80-4eee-a1ad-353a713f67d0 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:05.803045 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d46924-a279-48a3-8343-0d272c5e66d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:05.803681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d46924-a279-48a3-8343-0d272c5e66d4 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:05.803954 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d46924-a279-48a3-8343-0d272c5e66d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:05.804222 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d46924-a279-48a3-8343-0d272c5e66d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:05.811136 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d46924-a279-48a3-8343-0d272c5e66d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:05.812666 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d46924-a279-48a3-8343-0d272c5e66d4 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:05.823778 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef97de8-38c8-4110-9487-39b8dbc2ebf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:05.823924 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-46d46924-a279-48a3-8343-0d272c5e66d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:05.823924 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:05.823924 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:05.823924 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:05.823924 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:05.823924 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:05.823924 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:05.823924 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:05.823924 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:05.823924 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:05.823924 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:05.823924 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:05.823924 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:24:05.823924 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:24:05.823924 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:24:05.824792 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:24:05.824792 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:05.824792 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:05.824792 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:05.824792 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:05.824792 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:05.824792 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:05.824792 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:05.824792 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:05.824792 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:05.824792 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:05.824792 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:05.824792 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:05.824792 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:05.824792 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:05.824792 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:05.824792 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:05.825747 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:24:05.825747 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:24:05.825747 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:24:05.825747 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:24:05.825747 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:24:05.825747 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:24:05.825747 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:05.825747 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:05.828758 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef97de8-38c8-4110-9487-39b8dbc2ebf6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:05.830406 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 350/698] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:24:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:05.831266 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d46924-a279-48a3-8343-0d272c5e66d4 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:05.873188 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:05.899901 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:05.938171 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d46924-a279-48a3-8343-0d272c5e66d4 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:05.942584 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 349/699] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:24:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:24:05.943754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef97de8-38c8-4110-9487-39b8dbc2ebf6 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:05.980636 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a86b9a-7175-4296-842f-ea48d3820253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:05.980636 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a86b9a-7175-4296-842f-ea48d3820253 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:05.981366 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a86b9a-7175-4296-842f-ea48d3820253 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:05.981366 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a86b9a-7175-4296-842f-ea48d3820253 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:05.993471 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a86b9a-7175-4296-842f-ea48d3820253 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:05.994969 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a86b9a-7175-4296-842f-ea48d3820253 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:06.009677 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-13a86b9a-7175-4296-842f-ea48d3820253 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:06.009677 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:06.009677 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:06.009677 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:06.009677 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:06.009677 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:06.009677 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:06.009677 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:06.009677 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:06.009677 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:06.009677 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:06.009677 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:06.009677 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:06.009677 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:06.009677 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:06.009677 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:06.010673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:06.010673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:06.010673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:06.010673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:06.010673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:06.010673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:06.010673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:06.010673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:06.010673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:06.010673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:06.010673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:06.010673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:06.010673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:06.010673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:06.010673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:06.010673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:06.010673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:06.011564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:06.011564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:06.011564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:06.015696 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 351/700] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:06.016577 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a86b9a-7175-4296-842f-ea48d3820253 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:06.054862 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef97de8-38c8-4110-9487-39b8dbc2ebf6 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:06.093708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e35309-3a1e-459b-b162-d628db96acc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:06.094668 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e35309-3a1e-459b-b162-d628db96acc6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:06.095192 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e35309-3a1e-459b-b162-d628db96acc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:06.095661 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e35309-3a1e-459b-b162-d628db96acc6 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:06.096551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e35309-3a1e-459b-b162-d628db96acc6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:06.097053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e35309-3a1e-459b-b162-d628db96acc6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:06.098986 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e35309-3a1e-459b-b162-d628db96acc6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:06.106693 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:06.112699 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 350/701] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:24:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:06.118343 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e35309-3a1e-459b-b162-d628db96acc6 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:06.126630 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a86b9a-7175-4296-842f-ea48d3820253 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:06.145662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:06.165199 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b20855-4298-47de-8890-57165b632603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:06.166194 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b20855-4298-47de-8890-57165b632603 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:06.166653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b20855-4298-47de-8890-57165b632603 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:06.167062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b20855-4298-47de-8890-57165b632603 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:06.180660 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b20855-4298-47de-8890-57165b632603 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:06.182447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b20855-4298-47de-8890-57165b632603 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:06.199796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-22b20855-4298-47de-8890-57165b632603 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:06.199796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:06.199796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:06.199796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:06.199796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:06.199796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:06.199796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:06.199796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:06.199796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:06.199796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:06.199796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:06.199796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:06.199796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:06.199796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:06.199796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:06.199796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:06.200936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:06.200936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:06.200936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:06.200936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:06.200936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:06.200936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:06.200936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:06.200936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:06.200936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:06.200936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:06.200936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:06.200936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:06.200936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:06.200936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:06.200936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:06.200936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:06.200936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:06.201852 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:06.201852 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:06.201852 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:06.208477 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 352/702] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:06.209651 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b20855-4298-47de-8890-57165b632603 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:06.230733 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e35309-3a1e-459b-b162-d628db96acc6 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:06.269114 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b97fcc-e09a-4b3a-b592-15b0e47d8cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:06.269520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b97fcc-e09a-4b3a-b592-15b0e47d8cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:06.269802 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b97fcc-e09a-4b3a-b592-15b0e47d8cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:06.270076 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b97fcc-e09a-4b3a-b592-15b0e47d8cb9 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:06.270722 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b97fcc-e09a-4b3a-b592-15b0e47d8cb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:06.271026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b97fcc-e09a-4b3a-b592-15b0e47d8cb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:06.272445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b97fcc-e09a-4b3a-b592-15b0e47d8cb9 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:06.290776 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 351/703] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:06.291681 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b97fcc-e09a-4b3a-b592-15b0e47d8cb9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:06.321054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b20855-4298-47de-8890-57165b632603 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:06.354125 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:06.356977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c23f24-b689-49a6-acd1-695687197214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:06.357609 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c23f24-b689-49a6-acd1-695687197214 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:06.358311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c23f24-b689-49a6-acd1-695687197214 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:06.358311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c23f24-b689-49a6-acd1-695687197214 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:06.359033 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c23f24-b689-49a6-acd1-695687197214 None admin] RBAC: Adding query filter params (role.id=402c3e6fe3364285bd72784a297db11e, scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4, user.id=e69c0e02ad944f2a85fc5318018e5cb0) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:06.359276 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c23f24-b689-49a6-acd1-695687197214 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:06.360628 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c23f24-b689-49a6-acd1-695687197214 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:06.369275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 353/704] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:24:06 2022] GET /identity/v3/role_assignments?user.id=e69c0e02ad944f2a85fc5318018e5cb0&scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4&role.id=402c3e6fe3364285bd72784a297db11e => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:06.369275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c23f24-b689-49a6-acd1-695687197214 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:06.410922 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:06.419199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b97fcc-e09a-4b3a-b592-15b0e47d8cb9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:06.461486 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb246d98-fc13-4060-8c83-24eb69336fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:06.462635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb246d98-fc13-4060-8c83-24eb69336fe1 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:06.463160 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb246d98-fc13-4060-8c83-24eb69336fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:06.463617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb246d98-fc13-4060-8c83-24eb69336fe1 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:06.464603 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb246d98-fc13-4060-8c83-24eb69336fe1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:06.465091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb246d98-fc13-4060-8c83-24eb69336fe1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:06.466839 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb246d98-fc13-4060-8c83-24eb69336fe1 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:06.485652 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 352/705] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:06.487696 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a86415-808a-4cda-a7f1-75f50858cfcf None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:06.488179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a86415-808a-4cda-a7f1-75f50858cfcf None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:06.488643 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a86415-808a-4cda-a7f1-75f50858cfcf None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:06.489855 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c23f24-b689-49a6-acd1-695687197214 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:06.490750 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 353/706] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:06.492539 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a86415-808a-4cda-a7f1-75f50858cfcf None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:06.527350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5fe6e6-8ee4-4a23-a605-d94f665097a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:06.528074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fe6e6-8ee4-4a23-a605-d94f665097a0 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:06.528350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fe6e6-8ee4-4a23-a605-d94f665097a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:06.528614 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fe6e6-8ee4-4a23-a605-d94f665097a0 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:06.529393 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5fe6e6-8ee4-4a23-a605-d94f665097a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:06.530877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5fe6e6-8ee4-4a23-a605-d94f665097a0 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:06.610840 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a86415-808a-4cda-a7f1-75f50858cfcf None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:06.629484 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:06.648216 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 354/707] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 08:24:06 2022] POST /identity/v3/users => generated 254 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:24:06.649308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b59710-2bcf-4d98-8950-1cae93b89ec4 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:06.649595 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b59710-2bcf-4d98-8950-1cae93b89ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:06.649865 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b59710-2bcf-4d98-8950-1cae93b89ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:06.649972 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc91ae45-02b5-430d-add1-b9bdd0703d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:06.650700 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91ae45-02b5-430d-add1-b9bdd0703d37 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:06.650988 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91ae45-02b5-430d-add1-b9bdd0703d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:06.651260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91ae45-02b5-430d-add1-b9bdd0703d37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:06.661508 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc91ae45-02b5-430d-add1-b9bdd0703d37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:06.663069 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc91ae45-02b5-430d-add1-b9bdd0703d37 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:06.672357 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-07b59710-2bcf-4d98-8950-1cae93b89ec4 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:06.673631 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-07b59710-2bcf-4d98-8950-1cae93b89ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:06.677686 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-dc91ae45-02b5-430d-add1-b9bdd0703d37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:06.677686 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:06.677686 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:06.677686 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:06.677686 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:06.677686 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:06.677686 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:06.677686 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:06.677686 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:06.677686 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:06.677686 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:06.677686 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:06.677686 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:24:06.677686 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:24:06.677686 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:24:06.678675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:24:06.678675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:06.678675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:06.678675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:06.678675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:06.678675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:06.678675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:06.678675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:06.678675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:06.678675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:06.678675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:06.678675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:06.678675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:06.678675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:06.678675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:06.678675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:06.678675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:06.680187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:24:06.680187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:24:06.680187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:24:06.680187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:24:06.680187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:24:06.680187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:24:06.680187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:06.680187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:06.680187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 354/708] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:24:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:24:06.684752 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc91ae45-02b5-430d-add1-b9bdd0703d37 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:06.807119 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 355/709] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:24:06.843155 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b59710-2bcf-4d98-8950-1cae93b89ec4 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:06.870447 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc91ae45-02b5-430d-add1-b9bdd0703d37 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:06.872993 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:06.920801 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6b2aac-1ecb-4d86-bde7-bfa86bd22c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:06.921761 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b2aac-1ecb-4d86-bde7-bfa86bd22c8b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:06.922196 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b2aac-1ecb-4d86-bde7-bfa86bd22c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:06.922613 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b2aac-1ecb-4d86-bde7-bfa86bd22c8b None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:06.923539 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6b2aac-1ecb-4d86-bde7-bfa86bd22c8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:06.924026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6b2aac-1ecb-4d86-bde7-bfa86bd22c8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:06.925710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6b2aac-1ecb-4d86-bde7-bfa86bd22c8b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:06.941237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 355/710] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:24:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:06.947395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6b2aac-1ecb-4d86-bde7-bfa86bd22c8b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:06.975382 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b59710-2bcf-4d98-8950-1cae93b89ec4 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:06.994389 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:07.027515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae7bf4-7243-4535-a74a-61eee389d5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:07.028155 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae7bf4-7243-4535-a74a-61eee389d5b7 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:07.028428 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae7bf4-7243-4535-a74a-61eee389d5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:07.028694 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae7bf4-7243-4535-a74a-61eee389d5b7 None admin] PATH_INFO: `/v3/services/network` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:07.030665 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae7bf4-7243-4535-a74a-61eee389d5b7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:07.031255 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae7bf4-7243-4535-a74a-61eee389d5b7 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:07.048117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-5aae7bf4-7243-4535-a74a-61eee389d5b7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 08:24:07.048117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:07.048117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 08:24:07.048117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 08:24:07.048117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 08:24:07.048117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 08:24:07.048117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 08:24:07.048117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:07.048117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 08:24:07.048117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:07.048117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:24:07.048117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:07.048117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:07.048117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:07.048117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:07.048117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:07.048117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:07.049692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:07.049692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:07.049692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:07.049692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:07.049692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:07.049692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:07.049692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 08:24:07.049692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 08:24:07.049692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 08:24:07.049692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 08:24:07.049692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:07.049692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:07.049692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:07.049692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:07.049692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:07.049692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:07.050557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:07.050557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:07.050557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:07.050557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:07.050557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:07.050557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:07.050557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:07.050557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:07.050557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:07.050557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:07.050557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 08:24:07.050557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:07.050557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 08:24:07.050557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:07.053177 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 356/711] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:24:06 2022] GET /identity/v3/services/network => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:24:07.058857 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae7bf4-7243-4535-a74a-61eee389d5b7 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:07.078873 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6b2aac-1ecb-4d86-bde7-bfa86bd22c8b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:07.116467 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da7209-54ba-4922-a393-ec67dea5e91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:07.117246 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da7209-54ba-4922-a393-ec67dea5e91c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:07.117639 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da7209-54ba-4922-a393-ec67dea5e91c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:07.118003 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da7209-54ba-4922-a393-ec67dea5e91c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:07.118944 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da7209-54ba-4922-a393-ec67dea5e91c None admin] RBAC: Adding query filter params (role.id=c1e514a4b5424f959b069f36cbfd8147, scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4, user.id=a3317f751ec74bde95ec262394001292) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:07.119219 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da7209-54ba-4922-a393-ec67dea5e91c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:07.120797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da7209-54ba-4922-a393-ec67dea5e91c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:07.127193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 356/712] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:24:06 2022] GET /identity/v3/role_assignments?user.id=a3317f751ec74bde95ec262394001292&scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4&role.id=c1e514a4b5424f959b069f36cbfd8147 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:07.161607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:07.179213 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae7bf4-7243-4535-a74a-61eee389d5b7 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:07.183697 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955ac05-76e4-47af-bd61-c26f28c5b0a5 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:07.184151 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955ac05-76e4-47af-bd61-c26f28c5b0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:07.184706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955ac05-76e4-47af-bd61-c26f28c5b0a5 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:07.186542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 357/713] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 08:24:07.193043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03ec4e-4eb6-484c-bc22-3b13314b821d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:07.193447 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03ec4e-4eb6-484c-bc22-3b13314b821d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:07.193788 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03ec4e-4eb6-484c-bc22-3b13314b821d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:07.229610 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-cc03ec4e-4eb6-484c-bc22-3b13314b821d None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:07.229610 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-cc03ec4e-4eb6-484c-bc22-3b13314b821d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:07.239689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749f440-c151-4983-b6f7-4ea3b8332319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:07.240314 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749f440-c151-4983-b6f7-4ea3b8332319 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:07.240590 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749f440-c151-4983-b6f7-4ea3b8332319 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:07.240870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749f440-c151-4983-b6f7-4ea3b8332319 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:07.241533 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1749f440-c151-4983-b6f7-4ea3b8332319 None admin] RBAC: Adding query filter params (name=network) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:07.241833 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1749f440-c151-4983-b6f7-4ea3b8332319 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:07.243238 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1749f440-c151-4983-b6f7-4ea3b8332319 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:07.251782 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 357/714] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Aug 8 08:24:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:07.259073 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749f440-c151-4983-b6f7-4ea3b8332319 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:07.356081 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 358/715] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:24:07.422328 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749f440-c151-4983-b6f7-4ea3b8332319 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:07.455310 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03ec4e-4eb6-484c-bc22-3b13314b821d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:07.489958 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c21380-97ce-47b5-b4bc-aab3bd77f98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:07.490815 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c21380-97ce-47b5-b4bc-aab3bd77f98f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:07.491421 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c21380-97ce-47b5-b4bc-aab3bd77f98f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:07.492693 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c21380-97ce-47b5-b4bc-aab3bd77f98f None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:07.493527 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c21380-97ce-47b5-b4bc-aab3bd77f98f None admin] RBAC: Adding query filter params (type=network) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:07.494337 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c21380-97ce-47b5-b4bc-aab3bd77f98f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:07.496557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c21380-97ce-47b5-b4bc-aab3bd77f98f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:07.506862 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 358/716] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Aug 8 08:24:07 2022] GET /identity/v3/services?type=network => generated 124 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:07.514626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:07.629316 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03ec4e-4eb6-484c-bc22-3b13314b821d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:07.683973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df9440-abf8-4be8-ac89-c62bbb4eba91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:07.684656 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df9440-abf8-4be8-ac89-c62bbb4eba91 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:07.685070 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df9440-abf8-4be8-ac89-c62bbb4eba91 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:07.685430 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df9440-abf8-4be8-ac89-c62bbb4eba91 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:07.686093 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df9440-abf8-4be8-ac89-c62bbb4eba91 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:07.688105 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df9440-abf8-4be8-ac89-c62bbb4eba91 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:07.704168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-49df9440-abf8-4be8-ac89-c62bbb4eba91 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 08:24:07.704168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:07.704168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 08:24:07.704168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 08:24:07.704168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 08:24:07.704168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 08:24:07.704168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 08:24:07.704168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:07.704168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 08:24:07.704168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:07.704168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:24:07.704168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:07.704168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:07.704168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:07.704168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:07.704168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:07.704168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:07.705284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:07.705284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:07.705284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:07.705284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:07.705284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:07.705284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:07.705284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 08:24:07.705284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 08:24:07.705284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 08:24:07.705284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 08:24:07.705284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:07.705284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:07.705284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:07.705284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:07.705284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:07.705284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:07.706136 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:07.706136 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:07.706136 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:07.706136 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:07.706136 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:07.706136 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:07.706136 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:07.706136 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:07.706136 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:07.706136 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:07.706136 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 08:24:07.706136 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:07.706136 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 08:24:07.706136 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:07.709510 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 359/717] 10.222.0.33 () {56 vars in 1129 bytes} [Mon Aug 8 08:24:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:24:07.719162 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c21380-97ce-47b5-b4bc-aab3bd77f98f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:07.735999 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:07.829726 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c21380-97ce-47b5-b4bc-aab3bd77f98f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:07.867160 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bfedda-3f04-4b6b-9fe6-f9d40e76162c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:07.868034 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bfedda-3f04-4b6b-9fe6-f9d40e76162c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:07.868450 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bfedda-3f04-4b6b-9fe6-f9d40e76162c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:07.868858 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bfedda-3f04-4b6b-9fe6-f9d40e76162c None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:07.869676 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bfedda-3f04-4b6b-9fe6-f9d40e76162c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:07.870132 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bfedda-3f04-4b6b-9fe6-f9d40e76162c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:07.871618 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bfedda-3f04-4b6b-9fe6-f9d40e76162c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:07.876637 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 359/718] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Aug 8 08:24:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:07.881936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df9440-abf8-4be8-ac89-c62bbb4eba91 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:08.019389 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:08.024944 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df9440-abf8-4be8-ac89-c62bbb4eba91 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:08.058790 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327dbd1-502e-45b9-927a-8db3a034d255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:08.059404 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327dbd1-502e-45b9-927a-8db3a034d255 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:08.059684 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327dbd1-502e-45b9-927a-8db3a034d255 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:08.059960 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327dbd1-502e-45b9-927a-8db3a034d255 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:08.060615 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327dbd1-502e-45b9-927a-8db3a034d255 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:08.061504 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327dbd1-502e-45b9-927a-8db3a034d255 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:08.063056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327dbd1-502e-45b9-927a-8db3a034d255 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:08.069065 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 360/719] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Aug 8 08:24:07 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:08.938343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ccf1ed-dc1c-435e-a515-ebc28cc09b55 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:08.938989 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ccf1ed-dc1c-435e-a515-ebc28cc09b55 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:08.938989 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ccf1ed-dc1c-435e-a515-ebc28cc09b55 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:08.940518 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 360/720] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:08.946241 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de8485-c987-49d9-b7e4-01f613629ce0 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:08.946708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de8485-c987-49d9-b7e4-01f613629ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:08.946795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de8485-c987-49d9-b7e4-01f613629ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:08.969002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c9de8485-c987-49d9-b7e4-01f613629ce0 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:08.970246 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c9de8485-c987-49d9-b7e4-01f613629ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:09.041444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 361/721] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:24:09.075478 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ccf1ed-dc1c-435e-a515-ebc28cc09b55 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:09.183332 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:09.211494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ccf1ed-dc1c-435e-a515-ebc28cc09b55 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:09.264177 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d9488-2352-4f07-b477-c0fa950cde60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:09.264894 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d9488-2352-4f07-b477-c0fa950cde60 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:09.265221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d9488-2352-4f07-b477-c0fa950cde60 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:09.265473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d9488-2352-4f07-b477-c0fa950cde60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:09.283669 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d9488-2352-4f07-b477-c0fa950cde60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:24:09.288343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-125d9488-2352-4f07-b477-c0fa950cde60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:24:09.288343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:09.288343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:09.288343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:09.288343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:09.288343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:09.288343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:09.288343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:09.288343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:09.288343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:09.288343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:09.288343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:09.288343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:24:09.288343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:24:09.288343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:24:09.288343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:24:09.289484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:24:09.289484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:24:09.289484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:24:09.289484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:24:09.289484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:09.289484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:09.289484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:09.289484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:09.289484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:09.289484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:09.289484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:09.289484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:09.289484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:09.289484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:09.289484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:09.289484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:09.290762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:09.290762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:09.290762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:09.290762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:09.290762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:24:09.290762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:24:09.290762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:24:09.290762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:24:09.290762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:24:09.290762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:24:09.290762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:24:09.290762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:09.290762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 361/722] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:24:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:09.298023 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de8485-c987-49d9-b7e4-01f613629ce0 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:09.401648 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:09.413858 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de8485-c987-49d9-b7e4-01f613629ce0 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:09.445645 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f3198-eba1-4b9d-99f6-2ff1e64b070d None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:09.446226 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f3198-eba1-4b9d-99f6-2ff1e64b070d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:09.446885 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f3198-eba1-4b9d-99f6-2ff1e64b070d None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:09.449672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 362/723] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 08:24:09.454779 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36043536-48c4-459c-8744-8f0a74240459 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:09.455651 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36043536-48c4-459c-8744-8f0a74240459 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:09.456144 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36043536-48c4-459c-8744-8f0a74240459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:09.476876 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-36043536-48c4-459c-8744-8f0a74240459 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:09.478008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-36043536-48c4-459c-8744-8f0a74240459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:09.565585 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 363/724] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:24:09.567173 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae3918-5ffe-4da6-ad57-e842c1769a69 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:09.567556 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae3918-5ffe-4da6-ad57-e842c1769a69 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:09.567833 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae3918-5ffe-4da6-ad57-e842c1769a69 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:09.569754 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 364/725] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:09.569948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36c9e2-b23b-4ba9-abd6-38922722df01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:09.570516 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36c9e2-b23b-4ba9-abd6-38922722df01 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:09.570787 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36c9e2-b23b-4ba9-abd6-38922722df01 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:09.571160 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36c9e2-b23b-4ba9-abd6-38922722df01 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:09.571839 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c36c9e2-b23b-4ba9-abd6-38922722df01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:09.572143 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c36c9e2-b23b-4ba9-abd6-38922722df01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:09.573609 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c36c9e2-b23b-4ba9-abd6-38922722df01 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:09.574986 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c7e36-1bc3-48c6-bada-acaca6b9bf8b None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:09.575272 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c7e36-1bc3-48c6-bada-acaca6b9bf8b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:09.575541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c7e36-1bc3-48c6-bada-acaca6b9bf8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:09.583246 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 362/726] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:24:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:09.588549 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36c9e2-b23b-4ba9-abd6-38922722df01 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:09.602637 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-a09c7e36-1bc3-48c6-bada-acaca6b9bf8b None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:09.603903 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a09c7e36-1bc3-48c6-bada-acaca6b9bf8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:09.676927 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:09.681480 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 365/727] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 08:24:09.682363 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c7e36-1bc3-48c6-bada-acaca6b9bf8b None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:09.694205 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36c9e2-b23b-4ba9-abd6-38922722df01 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:09.737218 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fadcd33-6b20-42c8-a0ae-faaccc7e9432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:09.737875 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fadcd33-6b20-42c8-a0ae-faaccc7e9432 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:09.738158 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fadcd33-6b20-42c8-a0ae-faaccc7e9432 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:09.738427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fadcd33-6b20-42c8-a0ae-faaccc7e9432 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:09.751580 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fadcd33-6b20-42c8-a0ae-faaccc7e9432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:09.753982 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fadcd33-6b20-42c8-a0ae-faaccc7e9432 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:09.774936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-2fadcd33-6b20-42c8-a0ae-faaccc7e9432 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:09.774936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:09.774936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:09.774936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:09.774936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:09.774936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:09.774936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:09.774936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:09.774936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:09.774936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:09.774936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:09.774936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:09.774936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:09.774936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:09.774936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:09.774936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:09.776145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:09.776145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:09.776145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:09.776145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:09.776145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:09.776145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:09.776145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:09.776145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:09.776145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:09.776145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:09.776145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:09.776145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:09.776145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:09.776145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:09.776145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:09.776145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:09.776145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:09.777416 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:09.777416 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:09.777416 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:09.777416 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 363/728] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:09.777416 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fadcd33-6b20-42c8-a0ae-faaccc7e9432 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:09.783234 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:09.809106 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c7e36-1bc3-48c6-bada-acaca6b9bf8b None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:09.848140 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d32561c-cca1-4d58-9085-216b3e4fab86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:09.849105 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32561c-cca1-4d58-9085-216b3e4fab86 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:09.849575 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32561c-cca1-4d58-9085-216b3e4fab86 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:09.849995 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32561c-cca1-4d58-9085-216b3e4fab86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:09.860746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d32561c-cca1-4d58-9085-216b3e4fab86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:24:09.865656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-5d32561c-cca1-4d58-9085-216b3e4fab86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:24:09.865656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:09.865656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:09.865656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:09.865656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:09.865656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:09.865656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:09.865656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:09.865656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:09.865656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:09.865656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:09.865656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:09.865656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:24:09.865656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:24:09.865656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:24:09.865656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:24:09.866670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:24:09.866670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:24:09.866670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:24:09.866670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:24:09.866670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:09.866670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:09.866670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:09.866670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:09.866670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:09.866670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:09.866670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:09.866670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:09.866670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:09.866670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:09.866670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:09.866670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:09.867515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:09.867515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:09.867515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:09.867515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:09.867515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:24:09.867515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:24:09.867515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:24:09.867515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:24:09.867515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:24:09.867515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:24:09.867515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:24:09.867515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:09.870116 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 366/729] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:24:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:09.871074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d32561c-cca1-4d58-9085-216b3e4fab86 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:09.907111 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fadcd33-6b20-42c8-a0ae-faaccc7e9432 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:09.938966 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:09.946705 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895de8fe-64ed-434a-a921-d3268e303a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:09.947426 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895de8fe-64ed-434a-a921-d3268e303a1a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:09.947763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895de8fe-64ed-434a-a921-d3268e303a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:09.948131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895de8fe-64ed-434a-a921-d3268e303a1a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:09.959245 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895de8fe-64ed-434a-a921-d3268e303a1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:24:09.964453 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-895de8fe-64ed-434a-a921-d3268e303a1a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:24:09.964453 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:09.964453 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:09.964453 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:09.964453 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:09.964453 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:09.964453 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:09.964453 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:09.964453 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:09.964453 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:09.964453 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:09.964453 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:09.964453 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:24:09.964453 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:24:09.964453 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:24:09.964453 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:24:09.965404 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:24:09.965404 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:24:09.965404 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:24:09.965404 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:24:09.965404 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:09.965404 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:09.965404 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:09.965404 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:09.965404 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:09.965404 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:09.965404 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:09.965404 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:09.965404 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:09.965404 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:09.965404 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:09.965404 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:09.966243 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:09.966243 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:09.966243 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:09.966243 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:09.966243 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:24:09.966243 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:24:09.966243 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:24:09.966243 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:24:09.966243 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:24:09.966243 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:24:09.966243 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:24:09.966243 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:09.967288 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 364/730] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 08:24:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:09.968781 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895de8fe-64ed-434a-a921-d3268e303a1a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:10.021433 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d32561c-cca1-4d58-9085-216b3e4fab86 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:10.042576 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:10.057116 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba7960-4240-4e8a-99a2-0ae8ae1d606d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:10.057895 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba7960-4240-4e8a-99a2-0ae8ae1d606d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:10.058255 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba7960-4240-4e8a-99a2-0ae8ae1d606d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:10.058618 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba7960-4240-4e8a-99a2-0ae8ae1d606d None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:10.059364 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba7960-4240-4e8a-99a2-0ae8ae1d606d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:10.059760 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba7960-4240-4e8a-99a2-0ae8ae1d606d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:10.061975 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba7960-4240-4e8a-99a2-0ae8ae1d606d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:10.074609 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 367/731] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:10.075740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba7960-4240-4e8a-99a2-0ae8ae1d606d None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:10.088028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895de8fe-64ed-434a-a921-d3268e303a1a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:10.126720 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5d427-fb94-4299-ade1-9752319883eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:10.127738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5d427-fb94-4299-ade1-9752319883eb None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:10.128183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5d427-fb94-4299-ade1-9752319883eb None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:10.128731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5d427-fb94-4299-ade1-9752319883eb None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:10.129626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce5d427-fb94-4299-ade1-9752319883eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:10.130093 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce5d427-fb94-4299-ade1-9752319883eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:10.131625 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce5d427-fb94-4299-ade1-9752319883eb None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:10.141203 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 365/732] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:24:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:10.142277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5d427-fb94-4299-ade1-9752319883eb None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:10.183460 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba7960-4240-4e8a-99a2-0ae8ae1d606d None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:10.202306 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:10.221535 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f532f85-fb0d-4ab7-8090-7bd762ce7d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:10.222147 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f532f85-fb0d-4ab7-8090-7bd762ce7d1f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:10.222425 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f532f85-fb0d-4ab7-8090-7bd762ce7d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:10.222697 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f532f85-fb0d-4ab7-8090-7bd762ce7d1f None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:10.223361 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f532f85-fb0d-4ab7-8090-7bd762ce7d1f None admin] RBAC: Adding query filter params (name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:10.224090 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f532f85-fb0d-4ab7-8090-7bd762ce7d1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:10.225933 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f532f85-fb0d-4ab7-8090-7bd762ce7d1f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:10.237287 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 368/733] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 08:24:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:10.238226 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f532f85-fb0d-4ab7-8090-7bd762ce7d1f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:10.252774 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5d427-fb94-4299-ade1-9752319883eb None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:10.281377 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:10.290281 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d7df7-e80f-4f19-975f-332375436534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:10.297672 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d7df7-e80f-4f19-975f-332375436534 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:10.297672 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d7df7-e80f-4f19-975f-332375436534 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:10.297672 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d7df7-e80f-4f19-975f-332375436534 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:10.300350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d7df7-e80f-4f19-975f-332375436534 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:10.301727 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d7df7-e80f-4f19-975f-332375436534 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:10.312795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-b72d7df7-e80f-4f19-975f-332375436534 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 08:24:10.312795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:10.312795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:10.312795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:10.312795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:10.312795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:10.312795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:10.312795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:10.312795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:10.312795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:10.312795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:10.312795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:10.312795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:24:10.312795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:24:10.312795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:24:10.312795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:24:10.313818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:10.313818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:10.313818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:24:10.313818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:10.313818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:24:10.313818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:10.313818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:10.313818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:10.313818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:10.313818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:10.313818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:10.313818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:10.313818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:10.313818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:10.313818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:10.313818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:10.313818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:10.313818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:10.319985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:10.319985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:10.319985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:24:10.319985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:24:10.319985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:24:10.319985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:24:10.319985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:24:10.319985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:24:10.319985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 08:24:10.319985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:10.319985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 366/734] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:24:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:10.319985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc1048-c4d1-4af4-8509-50c5d6da4e24 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:10.319985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc1048-c4d1-4af4-8509-50c5d6da4e24 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:10.321274 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc1048-c4d1-4af4-8509-50c5d6da4e24 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:10.321274 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 367/735] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:10.321274 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc1048-c4d1-4af4-8509-50c5d6da4e24 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:10.352628 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f532f85-fb0d-4ab7-8090-7bd762ce7d1f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:10.388390 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3823fa-428d-4d13-8826-472aefe9e71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:10.389116 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3823fa-428d-4d13-8826-472aefe9e71b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:10.389945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3823fa-428d-4d13-8826-472aefe9e71b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:10.390191 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3823fa-428d-4d13-8826-472aefe9e71b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:10.400883 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3823fa-428d-4d13-8826-472aefe9e71b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:10.402384 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3823fa-428d-4d13-8826-472aefe9e71b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:10.426522 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc1048-c4d1-4af4-8509-50c5d6da4e24 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:10.427730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-fb3823fa-428d-4d13-8826-472aefe9e71b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:10.427730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:10.427730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:10.427730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:10.427730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:10.427730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:10.427730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:10.427730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:10.427730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:10.427730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:10.427730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:10.427730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:10.427730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:10.427730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:10.427730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:10.427730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:10.428841 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:10.428841 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:10.428841 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:10.428841 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:10.428841 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:10.428841 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:10.428841 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:10.428841 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:10.428841 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:10.428841 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:10.428841 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:10.428841 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:10.428841 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:10.428841 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:10.428841 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:10.428841 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:10.428841 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:10.429963 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:10.429963 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:10.429963 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:10.429963 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 369/736] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:10.430304 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3823fa-428d-4d13-8826-472aefe9e71b None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:10.463821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:10.467068 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a630b-0f9e-4ebd-88e7-3f8fb785f510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:10.467731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a630b-0f9e-4ebd-88e7-3f8fb785f510 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:10.468008 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a630b-0f9e-4ebd-88e7-3f8fb785f510 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:10.468454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a630b-0f9e-4ebd-88e7-3f8fb785f510 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:10.484103 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a630b-0f9e-4ebd-88e7-3f8fb785f510 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:10.487970 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a630b-0f9e-4ebd-88e7-3f8fb785f510 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:10.500120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-159a630b-0f9e-4ebd-88e7-3f8fb785f510 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:10.500120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:10.500120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:10.500120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:10.500120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:10.500120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:10.500120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:10.500120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:10.500120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:10.500120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:10.500120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:10.500120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:10.500120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:10.500120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:10.500120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:10.500120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:10.501284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:10.501284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:10.501284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:10.501284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:10.501284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:10.501284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:10.501284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:10.501284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:10.501284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:10.501284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:10.501284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:10.501284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:10.501284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:10.501284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:10.501284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:10.501284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:10.501284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:10.502173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:10.502173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:10.502173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:10.502173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 368/737] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:10.502596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd1d4c-009c-44f6-9d7d-67ed7585597b None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:10.502673 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd1d4c-009c-44f6-9d7d-67ed7585597b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:10.502939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd1d4c-009c-44f6-9d7d-67ed7585597b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:10.520673 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-6ecd1d4c-009c-44f6-9d7d-67ed7585597b None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:10.522053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6ecd1d4c-009c-44f6-9d7d-67ed7585597b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:10.546971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3823fa-428d-4d13-8826-472aefe9e71b None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:10.555600 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:10.585137 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadaf60-811d-4180-bd52-0d7ae713c4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:10.585773 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadaf60-811d-4180-bd52-0d7ae713c4ce None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:10.586104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadaf60-811d-4180-bd52-0d7ae713c4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:10.586577 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadaf60-811d-4180-bd52-0d7ae713c4ce None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:10.587495 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feadaf60-811d-4180-bd52-0d7ae713c4ce None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:10.587916 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feadaf60-811d-4180-bd52-0d7ae713c4ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:10.589370 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feadaf60-811d-4180-bd52-0d7ae713c4ce None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:10.596603 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 369/738] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:24:10.597639 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd1d4c-009c-44f6-9d7d-67ed7585597b None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:10.620340 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 370/739] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Aug 8 08:24:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:10.620340 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadaf60-811d-4180-bd52-0d7ae713c4ce None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:10.706824 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:10.719019 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadaf60-811d-4180-bd52-0d7ae713c4ce None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:10.722405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd1d4c-009c-44f6-9d7d-67ed7585597b None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:10.753638 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b82d4-70a2-44a8-92ff-436178e9404a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:10.754246 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b82d4-70a2-44a8-92ff-436178e9404a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:10.754476 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8f827-0ed5-4849-8bc4-a4d65c202c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:10.754604 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b82d4-70a2-44a8-92ff-436178e9404a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:10.754777 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b82d4-70a2-44a8-92ff-436178e9404a None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:10.755279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8f827-0ed5-4849-8bc4-a4d65c202c23 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:10.755424 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216b82d4-70a2-44a8-92ff-436178e9404a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:10.755577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8f827-0ed5-4849-8bc4-a4d65c202c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:10.755695 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216b82d4-70a2-44a8-92ff-436178e9404a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:10.755826 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8f827-0ed5-4849-8bc4-a4d65c202c23 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:10.756447 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c8f827-0ed5-4849-8bc4-a4d65c202c23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:10.756747 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c8f827-0ed5-4849-8bc4-a4d65c202c23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:10.757060 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216b82d4-70a2-44a8-92ff-436178e9404a None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:10.758121 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c8f827-0ed5-4849-8bc4-a4d65c202c23 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:10.768491 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 371/740] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:10.769086 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b82d4-70a2-44a8-92ff-436178e9404a None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:10.769453 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 370/741] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:10.770251 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8f827-0ed5-4849-8bc4-a4d65c202c23 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:10.798575 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:10.867851 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8f827-0ed5-4849-8bc4-a4d65c202c23 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:10.870551 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b82d4-70a2-44a8-92ff-436178e9404a None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:10.900926 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f116614-e8bc-4ee7-a8aa-c41b2e46117d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:10.901552 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f116614-e8bc-4ee7-a8aa-c41b2e46117d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:10.901831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f116614-e8bc-4ee7-a8aa-c41b2e46117d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:10.902098 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f116614-e8bc-4ee7-a8aa-c41b2e46117d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:10.902565 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be339385-2664-4dcb-8a2e-272030849ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:10.903157 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be339385-2664-4dcb-8a2e-272030849ffd None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:10.903433 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be339385-2664-4dcb-8a2e-272030849ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:10.903702 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be339385-2664-4dcb-8a2e-272030849ffd None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:10.904339 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be339385-2664-4dcb-8a2e-272030849ffd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:10.906261 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be339385-2664-4dcb-8a2e-272030849ffd None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:10.981747 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 372/742] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Aug 8 08:24:10 2022] POST /identity/v3/services => generated 240 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:24:10.982980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0b703-3d48-4952-af54-9c23424a2feb None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:10.983327 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0b703-3d48-4952-af54-9c23424a2feb None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:10.983737 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0b703-3d48-4952-af54-9c23424a2feb None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:10.984934 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 373/743] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:10.985664 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b0b703-3d48-4952-af54-9c23424a2feb None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:10.998396 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:11.023399 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f116614-e8bc-4ee7-a8aa-c41b2e46117d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:11.024935 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f116614-e8bc-4ee7-a8aa-c41b2e46117d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:11.056486 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:11.115282 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b0b703-3d48-4952-af54-9c23424a2feb None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:11.140679 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 371/744] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:24:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 08:24:11.142223 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f116614-e8bc-4ee7-a8aa-c41b2e46117d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:11.163101 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1eab46-75a5-4ff1-9e82-a4efd5f77410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:11.163342 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1eab46-75a5-4ff1-9e82-a4efd5f77410 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:11.163433 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1eab46-75a5-4ff1-9e82-a4efd5f77410 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:11.163691 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1eab46-75a5-4ff1-9e82-a4efd5f77410 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:11.173009 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1eab46-75a5-4ff1-9e82-a4efd5f77410 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:11.174523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1eab46-75a5-4ff1-9e82-a4efd5f77410 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:11.187511 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-fe1eab46-75a5-4ff1-9e82-a4efd5f77410 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 08:24:11.187511 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:11.187511 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:11.187511 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:11.187511 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:11.187511 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:11.187511 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:11.187511 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:11.187511 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:11.187511 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:11.187511 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:11.187511 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:11.187511 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:24:11.187511 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:24:11.187511 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:24:11.187511 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:24:11.188481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:11.188481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:11.188481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:24:11.188481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:11.188481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:24:11.188481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:11.188481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:11.188481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:11.188481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:11.188481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:11.188481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:11.188481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:11.188481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:11.188481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:11.188481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:11.188481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:11.188481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:11.188481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:11.189515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:11.189515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:11.189515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:24:11.189515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:24:11.189515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:24:11.189515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:24:11.189515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:24:11.189515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:24:11.189515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 08:24:11.189515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:11.190050 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 374/745] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Aug 8 08:24:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:11.197187 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651ae8e-392a-4597-bf9d-bcf6deacbbd9 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:11.197187 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651ae8e-392a-4597-bf9d-bcf6deacbbd9 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:11.197187 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651ae8e-392a-4597-bf9d-bcf6deacbbd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:11.217798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-c651ae8e-392a-4597-bf9d-bcf6deacbbd9 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:11.219310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c651ae8e-392a-4597-bf9d-bcf6deacbbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:11.250238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f116614-e8bc-4ee7-a8aa-c41b2e46117d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:11.278310 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:11.284377 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:11.286595 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6cdd48-1a18-47ce-8d64-17026de95de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:11.287255 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cdd48-1a18-47ce-8d64-17026de95de2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:11.287667 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cdd48-1a18-47ce-8d64-17026de95de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:11.287840 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cdd48-1a18-47ce-8d64-17026de95de2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:11.292656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 375/746] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:11 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 08:24:11.294610 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651ae8e-392a-4597-bf9d-bcf6deacbbd9 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:11.303663 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6cdd48-1a18-47ce-8d64-17026de95de2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:11.306309 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6cdd48-1a18-47ce-8d64-17026de95de2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:11.320437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-9e6cdd48-1a18-47ce-8d64-17026de95de2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 08:24:11.320437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:11.320437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:11.320437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:11.320437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:11.320437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:11.320437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:11.320437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:11.320437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:11.320437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:11.320437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:11.320437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:11.320437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:24:11.320437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:24:11.320437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:24:11.320437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:24:11.321706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:11.321706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:11.321706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:24:11.321706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:11.321706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:24:11.321706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:11.321706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:11.321706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:11.321706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:11.321706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:11.321706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:11.321706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:11.321706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:11.321706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:11.321706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:11.321706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:11.321706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:11.321706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:11.322797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:11.322797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:11.322797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:24:11.322797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:24:11.322797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:24:11.322797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:24:11.322797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:24:11.322797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:24:11.322797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 08:24:11.322797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:11.329393 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 372/747] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Aug 8 08:24:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:11.330282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6cdd48-1a18-47ce-8d64-17026de95de2 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:11.400762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651ae8e-392a-4597-bf9d-bcf6deacbbd9 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:11.440361 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d77cd2-e5cf-46a7-aaa2-7d31a12cd703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:11.440995 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d77cd2-e5cf-46a7-aaa2-7d31a12cd703 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:11.441307 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d77cd2-e5cf-46a7-aaa2-7d31a12cd703 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:11.441579 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d77cd2-e5cf-46a7-aaa2-7d31a12cd703 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:11.444747 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6cdd48-1a18-47ce-8d64-17026de95de2 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:11.451962 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d77cd2-e5cf-46a7-aaa2-7d31a12cd703 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:11.453677 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d77cd2-e5cf-46a7-aaa2-7d31a12cd703 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:11.469974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-a9d77cd2-e5cf-46a7-aaa2-7d31a12cd703 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:11.469974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:11.469974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:11.469974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:11.469974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:11.469974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:11.469974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:11.469974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:11.469974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:11.469974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:11.469974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:11.469974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:11.469974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:11.469974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:11.469974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:11.469974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:11.470936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:11.470936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:11.470936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:11.470936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:11.470936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:11.470936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:11.470936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:11.470936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:11.470936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:11.470936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:11.470936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:11.470936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:11.470936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:11.470936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:11.470936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:11.470936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:11.470936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:11.471827 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:11.471827 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:11.471827 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:11.471827 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 376/748] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:11.471827 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d77cd2-e5cf-46a7-aaa2-7d31a12cd703 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:11.492379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21510cfb-0ce3-4a86-934b-085ca156b575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:11.493007 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21510cfb-0ce3-4a86-934b-085ca156b575 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:11.493322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21510cfb-0ce3-4a86-934b-085ca156b575 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:11.493596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21510cfb-0ce3-4a86-934b-085ca156b575 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:11.494354 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21510cfb-0ce3-4a86-934b-085ca156b575 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:11.494658 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21510cfb-0ce3-4a86-934b-085ca156b575 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:11.496150 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21510cfb-0ce3-4a86-934b-085ca156b575 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:11.516951 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 373/749] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Aug 8 08:24:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:11.517872 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21510cfb-0ce3-4a86-934b-085ca156b575 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:11.572603 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:11.630073 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21510cfb-0ce3-4a86-934b-085ca156b575 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:11.642324 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:11.664675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280bb04-1ebe-4562-958e-64a8e3744519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:11.665390 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280bb04-1ebe-4562-958e-64a8e3744519 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:11.665684 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280bb04-1ebe-4562-958e-64a8e3744519 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:11.666017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280bb04-1ebe-4562-958e-64a8e3744519 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:11.666794 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5280bb04-1ebe-4562-958e-64a8e3744519 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:11.667163 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5280bb04-1ebe-4562-958e-64a8e3744519 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:11.668575 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5280bb04-1ebe-4562-958e-64a8e3744519 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:11.685459 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 374/750] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Aug 8 08:24:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:11.686335 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280bb04-1ebe-4562-958e-64a8e3744519 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:11.707297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d77cd2-e5cf-46a7-aaa2-7d31a12cd703 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:11.742992 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65841146-d6a6-4a05-87d9-a7a2cc462e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:11.744001 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65841146-d6a6-4a05-87d9-a7a2cc462e33 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:11.744595 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65841146-d6a6-4a05-87d9-a7a2cc462e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:11.745049 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65841146-d6a6-4a05-87d9-a7a2cc462e33 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:11.745897 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65841146-d6a6-4a05-87d9-a7a2cc462e33 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:11.747803 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65841146-d6a6-4a05-87d9-a7a2cc462e33 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:11.787506 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280bb04-1ebe-4562-958e-64a8e3744519 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:11.789780 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:11.817235 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 377/751] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Aug 8 08:24:11 2022] POST /identity/v3/services => generated 251 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:24:11.825491 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040eb21c-d22d-452b-852e-ada850233f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:11.825934 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040eb21c-d22d-452b-852e-ada850233f19 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:11.826215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040eb21c-d22d-452b-852e-ada850233f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:11.826505 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040eb21c-d22d-452b-852e-ada850233f19 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:11.827871 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040eb21c-d22d-452b-852e-ada850233f19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:11.828073 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65841146-d6a6-4a05-87d9-a7a2cc462e33 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:11.828173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040eb21c-d22d-452b-852e-ada850233f19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:11.832643 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040eb21c-d22d-452b-852e-ada850233f19 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:11.848664 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 375/752] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:11.849764 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040eb21c-d22d-452b-852e-ada850233f19 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:11.968646 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65841146-d6a6-4a05-87d9-a7a2cc462e33 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:12.004179 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:12.038345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040eb21c-d22d-452b-852e-ada850233f19 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:12.046980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d98210-72b7-4633-a3fe-808084119ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:12.047903 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d98210-72b7-4633-a3fe-808084119ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:12.048352 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d98210-72b7-4633-a3fe-808084119ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:12.049324 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d98210-72b7-4633-a3fe-808084119ed6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:12.076170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1e0e9-de18-4c91-8848-ee6fd2d28d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:12.076729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1e0e9-de18-4c91-8848-ee6fd2d28d78 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:12.077037 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1e0e9-de18-4c91-8848-ee6fd2d28d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:12.077342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1e0e9-de18-4c91-8848-ee6fd2d28d78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:12.089650 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d1e0e9-de18-4c91-8848-ee6fd2d28d78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:12.091104 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d1e0e9-de18-4c91-8848-ee6fd2d28d78 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:12.106384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-a1d1e0e9-de18-4c91-8848-ee6fd2d28d78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:12.106384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:12.106384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:12.106384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:12.106384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:12.106384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:12.106384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:12.106384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:12.106384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:12.106384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:12.106384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:12.106384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:12.106384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:12.106384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:12.106384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:12.106384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:12.107409 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:12.107409 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:12.107409 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:12.107409 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:12.107409 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:12.107409 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:12.107409 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:12.107409 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:12.107409 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:12.107409 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:12.107409 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:12.107409 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:12.107409 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:12.107409 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:12.107409 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:12.107409 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:12.107409 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:12.108473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:12.108473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:12.108473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:12.108473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 376/753] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:12.109386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1e0e9-de18-4c91-8848-ee6fd2d28d78 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:12.127637 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:12.146924 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d98210-72b7-4633-a3fe-808084119ed6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:12.148911 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d98210-72b7-4633-a3fe-808084119ed6 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:12.239371 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1e0e9-de18-4c91-8848-ee6fd2d28d78 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:12.268563 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 378/754] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:24:11 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 08:24:12.270137 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d98210-72b7-4633-a3fe-808084119ed6 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:12.282217 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:12.284468 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce752eaf-d475-47da-b1ff-eba2f751b3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:12.285108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce752eaf-d475-47da-b1ff-eba2f751b3a6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:12.285431 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce752eaf-d475-47da-b1ff-eba2f751b3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:12.285991 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce752eaf-d475-47da-b1ff-eba2f751b3a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:12.293456 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce752eaf-d475-47da-b1ff-eba2f751b3a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:12.300462 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce752eaf-d475-47da-b1ff-eba2f751b3a6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:12.307215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-ce752eaf-d475-47da-b1ff-eba2f751b3a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:12.307215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:12.307215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:12.307215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:12.307215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:12.307215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:12.307215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:12.307215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:12.307215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:12.307215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:12.307215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:12.307215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:12.307215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:24:12.307215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:24:12.307215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:24:12.308108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:24:12.308108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:12.308108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:12.308108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:12.308108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:12.308108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:12.308108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:12.308108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:12.308108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:12.308108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:12.308108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:12.308108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:12.308108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:12.308108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:12.308108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:12.308108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:12.308108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:12.309002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:24:12.309002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:24:12.309002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:24:12.309002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:24:12.309002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:24:12.309002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:24:12.309002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:12.309002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:12.311897 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 377/755] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:24:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:12.312802 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce752eaf-d475-47da-b1ff-eba2f751b3a6 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:12.386723 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:12.404349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d98210-72b7-4633-a3fe-808084119ed6 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:12.427901 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce752eaf-d475-47da-b1ff-eba2f751b3a6 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:12.447962 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13482597-88cd-4017-8f89-33a83780c95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:12.448575 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13482597-88cd-4017-8f89-33a83780c95c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:12.448856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13482597-88cd-4017-8f89-33a83780c95c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:12.449125 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13482597-88cd-4017-8f89-33a83780c95c None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:12.449809 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13482597-88cd-4017-8f89-33a83780c95c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:12.450105 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13482597-88cd-4017-8f89-33a83780c95c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:12.452791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13482597-88cd-4017-8f89-33a83780c95c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:12.466743 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952c582-f2e9-407a-955e-8dabe014b5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:12.467929 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952c582-f2e9-407a-955e-8dabe014b5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:12.472433 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 379/756] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:12.472433 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13482597-88cd-4017-8f89-33a83780c95c None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:12.473114 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952c582-f2e9-407a-955e-8dabe014b5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:12.473674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952c582-f2e9-407a-955e-8dabe014b5f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:12.488058 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952c582-f2e9-407a-955e-8dabe014b5f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:12.489810 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952c582-f2e9-407a-955e-8dabe014b5f8 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:12.507754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-e952c582-f2e9-407a-955e-8dabe014b5f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:12.507754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:12.507754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:12.507754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:12.507754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:12.507754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:12.507754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:12.507754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:12.507754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:12.507754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:12.507754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:12.507754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:12.507754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:12.507754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:12.507754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:12.507754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:12.508748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:12.508748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:12.508748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:12.508748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:12.508748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:12.508748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:12.508748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:12.508748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:12.508748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:12.508748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:12.508748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:12.508748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:12.508748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:12.508748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:12.508748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:12.508748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:12.508748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:12.509727 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:12.509727 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:12.509727 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:12.513330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 378/757] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:12.514197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952c582-f2e9-407a-955e-8dabe014b5f8 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:12.542295 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:12.579191 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13482597-88cd-4017-8f89-33a83780c95c None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:12.615022 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e01127-a6d8-46f8-8561-c874981a71ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:12.615988 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e01127-a6d8-46f8-8561-c874981a71ff None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:12.616492 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e01127-a6d8-46f8-8561-c874981a71ff None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:12.616960 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e01127-a6d8-46f8-8561-c874981a71ff None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:12.617959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e01127-a6d8-46f8-8561-c874981a71ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:12.618507 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e01127-a6d8-46f8-8561-c874981a71ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:12.620129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e01127-a6d8-46f8-8561-c874981a71ff None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:12.624499 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952c582-f2e9-407a-955e-8dabe014b5f8 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:12.640059 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 380/758] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:24:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:12.641284 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e01127-a6d8-46f8-8561-c874981a71ff None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:12.649988 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:12.663382 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99cb08-981e-4776-b5a3-14644511a217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:12.666512 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99cb08-981e-4776-b5a3-14644511a217 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:12.666512 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99cb08-981e-4776-b5a3-14644511a217 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:12.666512 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99cb08-981e-4776-b5a3-14644511a217 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:12.673256 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99cb08-981e-4776-b5a3-14644511a217 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:12.675017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99cb08-981e-4776-b5a3-14644511a217 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:12.687364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-4a99cb08-981e-4776-b5a3-14644511a217 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:12.687364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:12.687364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:12.687364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:12.687364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:12.687364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:12.687364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:12.687364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:12.687364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:12.687364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:12.687364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:12.687364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:12.687364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:24:12.687364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:24:12.687364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:24:12.688282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:24:12.688282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:12.688282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:12.688282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:12.688282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:12.688282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:12.688282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:12.688282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:12.688282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:12.688282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:12.688282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:12.688282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:12.688282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:12.688282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:12.688282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:12.688282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:12.688282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:12.689189 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:24:12.689189 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:24:12.689189 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:24:12.689189 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:24:12.689189 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:24:12.689189 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:24:12.689189 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:12.689189 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:12.692193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 379/759] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:24:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:12.693236 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99cb08-981e-4776-b5a3-14644511a217 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:12.751853 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e01127-a6d8-46f8-8561-c874981a71ff None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:12.784927 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:12.792832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79dd87-e867-49b1-a834-8a3770cbdb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:12.793964 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79dd87-e867-49b1-a834-8a3770cbdb39 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:12.794710 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79dd87-e867-49b1-a834-8a3770cbdb39 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:12.795641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79dd87-e867-49b1-a834-8a3770cbdb39 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:12.797346 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79dd87-e867-49b1-a834-8a3770cbdb39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:12.797859 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79dd87-e867-49b1-a834-8a3770cbdb39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:12.798880 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99cb08-981e-4776-b5a3-14644511a217 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:12.799655 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f79dd87-e867-49b1-a834-8a3770cbdb39 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:12.818489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 381/760] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:12.818489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79dd87-e867-49b1-a834-8a3770cbdb39 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:12.848230 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da65a0a1-81bd-4bd7-9e8a-e71f9c763065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:12.849078 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65a0a1-81bd-4bd7-9e8a-e71f9c763065 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:12.849593 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65a0a1-81bd-4bd7-9e8a-e71f9c763065 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:12.850003 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65a0a1-81bd-4bd7-9e8a-e71f9c763065 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:12.850948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da65a0a1-81bd-4bd7-9e8a-e71f9c763065 None admin] RBAC: Adding query filter params (role.id=c1e514a4b5424f959b069f36cbfd8147, scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4, user.id=e69c0e02ad944f2a85fc5318018e5cb0) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:12.851390 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da65a0a1-81bd-4bd7-9e8a-e71f9c763065 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:12.853116 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da65a0a1-81bd-4bd7-9e8a-e71f9c763065 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:12.860261 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 380/761] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:24:12 2022] GET /identity/v3/role_assignments?user.id=e69c0e02ad944f2a85fc5318018e5cb0&scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4&role.id=c1e514a4b5424f959b069f36cbfd8147 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:12.861302 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da65a0a1-81bd-4bd7-9e8a-e71f9c763065 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:12.917110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:12.975601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da65a0a1-81bd-4bd7-9e8a-e71f9c763065 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:12.984215 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79dd87-e867-49b1-a834-8a3770cbdb39 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:13.018945 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9578d-a782-421b-ae6b-e74ee2b6661c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:13.019826 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9578d-a782-421b-ae6b-e74ee2b6661c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:13.020243 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9578d-a782-421b-ae6b-e74ee2b6661c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:13.020658 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9578d-a782-421b-ae6b-e74ee2b6661c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:13.033325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9578d-a782-421b-ae6b-e74ee2b6661c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:13.035489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676adc3d-69fd-49d2-bcda-d2b6cf733192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:13.036108 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676adc3d-69fd-49d2-bcda-d2b6cf733192 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:13.036328 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9578d-a782-421b-ae6b-e74ee2b6661c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:13.036430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676adc3d-69fd-49d2-bcda-d2b6cf733192 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:13.036642 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676adc3d-69fd-49d2-bcda-d2b6cf733192 None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:13.037376 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676adc3d-69fd-49d2-bcda-d2b6cf733192 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:13.037687 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676adc3d-69fd-49d2-bcda-d2b6cf733192 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:13.039127 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676adc3d-69fd-49d2-bcda-d2b6cf733192 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:13.050844 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-52a9578d-a782-421b-ae6b-e74ee2b6661c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:13.050844 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:13.050844 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:13.050844 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:13.050844 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:13.050844 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:13.050844 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:13.050844 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:13.050844 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:13.050844 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:13.050844 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:13.050844 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:13.050844 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:24:13.050844 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:24:13.050844 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:24:13.052126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:24:13.052126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:13.052126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:13.052126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:13.052126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:13.052126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:13.052126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:13.052126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:13.052126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:13.052126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:13.052126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:13.052126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:13.052126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:13.052126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:13.052126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:13.052126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:13.052126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:13.053003 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:24:13.053003 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:24:13.053003 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:24:13.053003 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:24:13.053003 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:24:13.053003 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:24:13.053003 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:13.053003 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:13.055673 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 381/763] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:24:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:13.057283 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43883a11-7b29-4bcb-9869-bb266d7092d5 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:13.057992 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 382/763] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:24:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:13.058274 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43883a11-7b29-4bcb-9869-bb266d7092d5 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:13.058741 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43883a11-7b29-4bcb-9869-bb266d7092d5 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:13.058935 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676adc3d-69fd-49d2-bcda-d2b6cf733192 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:13.065186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 382/764] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:13.066615 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43883a11-7b29-4bcb-9869-bb266d7092d5 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:13.110119 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:13.165074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676adc3d-69fd-49d2-bcda-d2b6cf733192 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:13.180089 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43883a11-7b29-4bcb-9869-bb266d7092d5 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:13.189772 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:13.199773 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678d97d-d32c-417e-b55e-908e8ae14cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:13.202482 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678d97d-d32c-417e-b55e-908e8ae14cf4 None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:13.202482 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678d97d-d32c-417e-b55e-908e8ae14cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:13.202482 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678d97d-d32c-417e-b55e-908e8ae14cf4 None admin] PATH_INFO: `/v3/projects/71e5e82c9f454cdd91dfeac5a57968c4/users/a3317f751ec74bde95ec262394001292/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:13.204037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678d97d-d32c-417e-b55e-908e8ae14cf4 None admin] RBAC: Authorizing `identity:create_grant(project_id=71e5e82c9f454cdd91dfeac5a57968c4, user_id=a3317f751ec74bde95ec262394001292, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:13.205739 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678d97d-d32c-417e-b55e-908e8ae14cf4 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:13.218296 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff39978-41af-4a06-bcf0-e5c134aa2731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:13.219249 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff39978-41af-4a06-bcf0-e5c134aa2731 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:13.219613 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff39978-41af-4a06-bcf0-e5c134aa2731 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:13.219885 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff39978-41af-4a06-bcf0-e5c134aa2731 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:13.220623 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff39978-41af-4a06-bcf0-e5c134aa2731 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:13.220975 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff39978-41af-4a06-bcf0-e5c134aa2731 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:13.222621 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff39978-41af-4a06-bcf0-e5c134aa2731 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:13.240879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 383/765] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:24:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:13.240879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e4f00-15aa-49fa-bbfb-cf4b111e73b0 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:13.240879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e4f00-15aa-49fa-bbfb-cf4b111e73b0 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:13.240879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e4f00-15aa-49fa-bbfb-cf4b111e73b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:13.257583 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-131e4f00-15aa-49fa-bbfb-cf4b111e73b0 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:13.258849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-131e4f00-15aa-49fa-bbfb-cf4b111e73b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:13.274324 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 383/766] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 08:24:13 2022] PUT /identity/v3/projects/71e5e82c9f454cdd91dfeac5a57968c4/users/a3317f751ec74bde95ec262394001292/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:24:13.282851 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678d97d-d32c-417e-b55e-908e8ae14cf4 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:13.348731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 384/767] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:13.374814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e4f00-15aa-49fa-bbfb-cf4b111e73b0 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:13.503590 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:13.508330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e4f00-15aa-49fa-bbfb-cf4b111e73b0 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:13.514217 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678d97d-d32c-417e-b55e-908e8ae14cf4 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:13.523260 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:13.564941 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8178f8-dc30-4684-9d7e-9bbd228a68bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:13.565617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc85b7-6782-4eb3-8e9e-0243683c47ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:13.566443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc85b7-6782-4eb3-8e9e-0243683c47ac None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:13.567200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8178f8-dc30-4684-9d7e-9bbd228a68bd None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:13.567621 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc85b7-6782-4eb3-8e9e-0243683c47ac None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:13.568043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8178f8-dc30-4684-9d7e-9bbd228a68bd None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:13.568450 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc85b7-6782-4eb3-8e9e-0243683c47ac None admin] PATH_INFO: `/v3/services/network` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:13.568862 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8178f8-dc30-4684-9d7e-9bbd228a68bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:13.569622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc85b7-6782-4eb3-8e9e-0243683c47ac None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:13.570505 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8178f8-dc30-4684-9d7e-9bbd228a68bd None admin] RBAC: Adding query filter params (role.id=402c3e6fe3364285bd72784a297db11e, scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4, user.id=7f3422c6f9a041ceba851b71bd15d0df) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:13.572034 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc85b7-6782-4eb3-8e9e-0243683c47ac None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:13.572494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8178f8-dc30-4684-9d7e-9bbd228a68bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:13.574057 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8178f8-dc30-4684-9d7e-9bbd228a68bd None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:13.587673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 384/768] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:24:13 2022] GET /identity/v3/role_assignments?user.id=7f3422c6f9a041ceba851b71bd15d0df&scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4&role.id=402c3e6fe3364285bd72784a297db11e => generated 259 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:13.590632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-39bc85b7-6782-4eb3-8e9e-0243683c47ac None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 08:24:13.590632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:13.590632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 08:24:13.590632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 08:24:13.590632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 08:24:13.590632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 08:24:13.590632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 08:24:13.590632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:13.590632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 08:24:13.590632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:13.590632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:24:13.590632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:13.590632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:13.590632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:13.590632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:13.590632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:13.590632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:13.591849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:13.591849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:13.591849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:13.591849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:13.591849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:13.591849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:13.591849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 08:24:13.591849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 08:24:13.591849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 08:24:13.591849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 08:24:13.591849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:13.591849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:13.591849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:13.591849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:13.591849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:13.591849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:13.593485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:13.593485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:13.593485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:13.593485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:13.593485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:13.593485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:13.593485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:13.593485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:13.593485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:13.593485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:13.593485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 08:24:13.593485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:13.593485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 08:24:13.593485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:13.596080 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 385/769] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:24:13 2022] GET /identity/v3/services/network => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:24:13.602338 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8178f8-dc30-4684-9d7e-9bbd228a68bd None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:13.717075 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8178f8-dc30-4684-9d7e-9bbd228a68bd None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:13.754638 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03306b5b-83ed-4c5f-956e-9f2ecad51244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:13.755636 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03306b5b-83ed-4c5f-956e-9f2ecad51244 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:13.756062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03306b5b-83ed-4c5f-956e-9f2ecad51244 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:13.756494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03306b5b-83ed-4c5f-956e-9f2ecad51244 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:13.757333 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03306b5b-83ed-4c5f-956e-9f2ecad51244 None admin] RBAC: Adding query filter params (name=network) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:13.757784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03306b5b-83ed-4c5f-956e-9f2ecad51244 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:13.759339 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03306b5b-83ed-4c5f-956e-9f2ecad51244 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:13.765604 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 385/770] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Aug 8 08:24:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:13.771592 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc85b7-6782-4eb3-8e9e-0243683c47ac None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:13.885590 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc85b7-6782-4eb3-8e9e-0243683c47ac None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:13.931662 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10da42c-06e1-44ba-a44a-4a76413da3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:13.932514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10da42c-06e1-44ba-a44a-4a76413da3cc None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:13.932938 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10da42c-06e1-44ba-a44a-4a76413da3cc None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:13.933447 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10da42c-06e1-44ba-a44a-4a76413da3cc None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:13.934241 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10da42c-06e1-44ba-a44a-4a76413da3cc None admin] RBAC: Adding query filter params (type=network) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:13.934696 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10da42c-06e1-44ba-a44a-4a76413da3cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:13.936364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10da42c-06e1-44ba-a44a-4a76413da3cc None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:13.945960 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 386/771] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Aug 8 08:24:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:13.952430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03306b5b-83ed-4c5f-956e-9f2ecad51244 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:14.001752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:14.087495 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03306b5b-83ed-4c5f-956e-9f2ecad51244 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:14.111079 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808fcfb-7a65-43c8-bf05-4089eb35d2f4 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:14.111583 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808fcfb-7a65-43c8-bf05-4089eb35d2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:14.111827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808fcfb-7a65-43c8-bf05-4089eb35d2f4 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:14.113485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 387/772] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 08:24:14.118489 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe935315-0142-4f10-9daf-9607f18d80b7 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:14.118773 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe935315-0142-4f10-9daf-9607f18d80b7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:14.119045 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe935315-0142-4f10-9daf-9607f18d80b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:14.130741 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed6b0d-d411-422b-a966-0d873e892dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:14.131354 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed6b0d-d411-422b-a966-0d873e892dcb None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:14.131625 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed6b0d-d411-422b-a966-0d873e892dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:14.131891 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed6b0d-d411-422b-a966-0d873e892dcb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:14.132592 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed6b0d-d411-422b-a966-0d873e892dcb None admin] RBAC: Adding query filter params (interface=public, service_id=f1a891802ed643d191e3d560d1ca15c1, region_id=RegionOne) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:14.132890 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed6b0d-d411-422b-a966-0d873e892dcb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:14.134269 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed6b0d-d411-422b-a966-0d873e892dcb None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:14.141097 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-fe935315-0142-4f10-9daf-9607f18d80b7 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:14.142328 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fe935315-0142-4f10-9daf-9607f18d80b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:14.144081 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 386/773] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Aug 8 08:24:13 2022] GET /identity/v3/endpoints?service_id=f1a891802ed643d191e3d560d1ca15c1&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:14.149384 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed6b0d-d411-422b-a966-0d873e892dcb None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:14.165265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:14.222672 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 388/774] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 08:24:14.245427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe935315-0142-4f10-9daf-9607f18d80b7 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:14.271456 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:14.275740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed6b0d-d411-422b-a966-0d873e892dcb None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:14.313437 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750a381-6858-4970-b82a-92da518c8d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:14.314049 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750a381-6858-4970-b82a-92da518c8d0c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:14.314385 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750a381-6858-4970-b82a-92da518c8d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:14.314659 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750a381-6858-4970-b82a-92da518c8d0c None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:14.315310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7750a381-6858-4970-b82a-92da518c8d0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:14.316670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7750a381-6858-4970-b82a-92da518c8d0c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:14.322372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 387/775] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Aug 8 08:24:14 2022] GET /identity/v3/services => generated 772 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:14.354892 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe935315-0142-4f10-9daf-9607f18d80b7 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:14.393217 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341f0ed-53ff-4e59-831f-c4575dba84d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:14.394251 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341f0ed-53ff-4e59-831f-c4575dba84d6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:14.394761 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341f0ed-53ff-4e59-831f-c4575dba84d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:14.395263 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341f0ed-53ff-4e59-831f-c4575dba84d6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:14.396065 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8341f0ed-53ff-4e59-831f-c4575dba84d6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:14.397725 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8341f0ed-53ff-4e59-831f-c4575dba84d6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:14.418710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-8341f0ed-53ff-4e59-831f-c4575dba84d6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 08:24:14.418710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:14.418710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 08:24:14.418710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 08:24:14.418710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 08:24:14.418710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 08:24:14.418710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 08:24:14.418710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:14.418710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 08:24:14.418710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:14.418710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:24:14.418710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:14.418710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:14.418710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:14.418710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:14.418710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:14.418710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:14.419688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:14.419688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:14.419688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:14.419688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:14.419688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:14.419688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:14.419688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 08:24:14.419688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 08:24:14.419688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 08:24:14.419688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 08:24:14.419688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:14.419688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:14.419688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:14.419688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:14.419688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:14.419688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:14.420551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:14.420551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:14.420551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:14.420551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:14.420551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:14.420551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:14.420551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:14.420551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:14.420551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:14.420551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:14.420551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 08:24:14.420551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:14.420551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 08:24:14.420551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:14.424591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 389/776] 10.222.0.33 () {56 vars in 1129 bytes} [Mon Aug 8 08:24:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:24:14.430960 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750a381-6858-4970-b82a-92da518c8d0c None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:14.580191 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750a381-6858-4970-b82a-92da518c8d0c None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:14.615194 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461256d1-fb32-4e72-a397-69d5393ac303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:14.615806 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461256d1-fb32-4e72-a397-69d5393ac303 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:14.616083 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461256d1-fb32-4e72-a397-69d5393ac303 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:14.616352 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461256d1-fb32-4e72-a397-69d5393ac303 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:14.616978 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461256d1-fb32-4e72-a397-69d5393ac303 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:14.617305 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461256d1-fb32-4e72-a397-69d5393ac303 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:14.618658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461256d1-fb32-4e72-a397-69d5393ac303 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:14.623587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 388/777] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Aug 8 08:24:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:14.629301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341f0ed-53ff-4e59-831f-c4575dba84d6 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:14.652767 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:14.769180 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341f0ed-53ff-4e59-831f-c4575dba84d6 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:14.814316 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4559f-960e-4e40-9b1c-2887fcb100b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:14.814934 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4559f-960e-4e40-9b1c-2887fcb100b2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:14.815217 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4559f-960e-4e40-9b1c-2887fcb100b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:14.815482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4559f-960e-4e40-9b1c-2887fcb100b2 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:14.816116 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf4559f-960e-4e40-9b1c-2887fcb100b2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:14.816414 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf4559f-960e-4e40-9b1c-2887fcb100b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:14.817806 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf4559f-960e-4e40-9b1c-2887fcb100b2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:14.829980 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 390/778] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Aug 8 08:24:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:14.836192 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461256d1-fb32-4e72-a397-69d5393ac303 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:14.887544 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:14.981616 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461256d1-fb32-4e72-a397-69d5393ac303 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:15.031498 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c225f-28c7-4999-bf16-fc0bbc934336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:15.032103 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c225f-28c7-4999-bf16-fc0bbc934336 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:15.032379 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c225f-28c7-4999-bf16-fc0bbc934336 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:15.032646 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c225f-28c7-4999-bf16-fc0bbc934336 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:15.033437 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c225f-28c7-4999-bf16-fc0bbc934336 None admin] RBAC: Adding query filter params (interface=public, service_id=409678241d97476ea1d3c06945f296e2, region_id=RegionOne) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:15.033740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c225f-28c7-4999-bf16-fc0bbc934336 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:15.035183 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c225f-28c7-4999-bf16-fc0bbc934336 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:15.042787 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 389/779] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Aug 8 08:24:14 2022] GET /identity/v3/endpoints?service_id=409678241d97476ea1d3c06945f296e2&interface=public®ion_id=RegionOne => generated 194 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:15.048779 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4559f-960e-4e40-9b1c-2887fcb100b2 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:15.175327 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:15.193655 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4559f-960e-4e40-9b1c-2887fcb100b2 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:15.249242 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d20dd0-1b93-408d-873d-b6f4dbca21d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:15.250383 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d20dd0-1b93-408d-873d-b6f4dbca21d6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:15.250900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d20dd0-1b93-408d-873d-b6f4dbca21d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:15.251376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d20dd0-1b93-408d-873d-b6f4dbca21d6 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:15.252215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d20dd0-1b93-408d-873d-b6f4dbca21d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:15.253822 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d20dd0-1b93-408d-873d-b6f4dbca21d6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:15.265634 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 391/780] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Aug 8 08:24:15 2022] GET /identity/v3/services => generated 772 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:15.296792 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f799fc-b4c3-4765-b6c8-f6713a878d93 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:15.297343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f799fc-b4c3-4765-b6c8-f6713a878d93 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:15.297343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f799fc-b4c3-4765-b6c8-f6713a878d93 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:15.299037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 390/781] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:15.304377 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1a0d6-070a-4f15-ab9f-6e7c9bf68051 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:15.304930 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1a0d6-070a-4f15-ab9f-6e7c9bf68051 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:15.305474 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1a0d6-070a-4f15-ab9f-6e7c9bf68051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:15.334326 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-62a1a0d6-070a-4f15-ab9f-6e7c9bf68051 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:15.335909 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-62a1a0d6-070a-4f15-ab9f-6e7c9bf68051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:15.424170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 392/782] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:15.461544 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f799fc-b4c3-4765-b6c8-f6713a878d93 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:15.574731 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:15.587033 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f799fc-b4c3-4765-b6c8-f6713a878d93 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:15.622943 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b24a62-8288-4566-b41c-066ff4787ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:15.624065 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b24a62-8288-4566-b41c-066ff4787ace None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:15.624565 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b24a62-8288-4566-b41c-066ff4787ace None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:15.625053 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b24a62-8288-4566-b41c-066ff4787ace None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:15.637096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b24a62-8288-4566-b41c-066ff4787ace None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:24:15.642684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-e1b24a62-8288-4566-b41c-066ff4787ace None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:24:15.642684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:15.642684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:15.642684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:15.642684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:15.642684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:15.642684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:15.642684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:15.642684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:15.642684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:15.642684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:15.642684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:15.642684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:24:15.642684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:24:15.642684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:24:15.642684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:24:15.643587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:24:15.643587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:24:15.643587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:24:15.643587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:24:15.643587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:15.643587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:15.643587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:15.643587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:15.643587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:15.643587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:15.643587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:15.643587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:15.643587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:15.643587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:15.643587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:15.643587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:15.644436 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:15.644436 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:15.644436 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:15.644436 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:15.644436 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:24:15.644436 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:24:15.644436 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:24:15.644436 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:24:15.644436 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:24:15.644436 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:24:15.644436 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:24:15.644436 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:15.647128 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 391/783] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:24:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:15.652612 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1a0d6-070a-4f15-ab9f-6e7c9bf68051 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:15.758772 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:15.785713 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1a0d6-070a-4f15-ab9f-6e7c9bf68051 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:15.832317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71642f6a-2803-4654-bbef-de66c37626a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:15.833265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71642f6a-2803-4654-bbef-de66c37626a4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:15.833541 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71642f6a-2803-4654-bbef-de66c37626a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:15.833805 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71642f6a-2803-4654-bbef-de66c37626a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:15.834466 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71642f6a-2803-4654-bbef-de66c37626a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:15.834757 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71642f6a-2803-4654-bbef-de66c37626a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:15.836169 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71642f6a-2803-4654-bbef-de66c37626a4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:15.840806 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdfd91-ab94-4573-acca-2a73ed928792 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:15.841092 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdfd91-ab94-4573-acca-2a73ed928792 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:15.841391 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdfd91-ab94-4573-acca-2a73ed928792 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:15.843591 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 392/784] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:15.845795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 393/785] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:24:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:15.849397 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be69be8-8045-4687-a019-17a26a36bfcd None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:15.849685 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be69be8-8045-4687-a019-17a26a36bfcd None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:15.849968 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be69be8-8045-4687-a019-17a26a36bfcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:15.851056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71642f6a-2803-4654-bbef-de66c37626a4 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:15.884964 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-6be69be8-8045-4687-a019-17a26a36bfcd None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:15.886191 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6be69be8-8045-4687-a019-17a26a36bfcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:16.004143 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 393/786] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:16.027561 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be69be8-8045-4687-a019-17a26a36bfcd None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:16.049773 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71642f6a-2803-4654-bbef-de66c37626a4 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:16.103563 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca3f060-812f-4b50-9c5d-c979e5106049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:16.104890 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3f060-812f-4b50-9c5d-c979e5106049 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:16.105412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3f060-812f-4b50-9c5d-c979e5106049 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:16.106031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3f060-812f-4b50-9c5d-c979e5106049 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:16.116439 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:16.125911 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca3f060-812f-4b50-9c5d-c979e5106049 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:16.128263 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca3f060-812f-4b50-9c5d-c979e5106049 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:16.154895 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-6ca3f060-812f-4b50-9c5d-c979e5106049 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:16.154895 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:16.154895 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:16.154895 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:16.154895 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:16.154895 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:16.154895 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:16.154895 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:16.154895 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:16.154895 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:16.154895 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:16.154895 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:16.154895 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:16.154895 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:16.154895 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:16.154895 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:16.155990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:16.155990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:16.155990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:16.155990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:16.155990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:16.155990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:16.155990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:16.155990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:16.155990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:16.155990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:16.155990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:16.155990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:16.155990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:16.155990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:16.155990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:16.155990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:16.155990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:16.156893 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:16.156893 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:16.156893 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:16.160843 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 394/787] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:16.168316 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca3f060-812f-4b50-9c5d-c979e5106049 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:16.191773 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:16.215595 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be69be8-8045-4687-a019-17a26a36bfcd None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:16.258140 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4902b5d-a310-4fd5-a0c6-7ae993177045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:16.258779 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4902b5d-a310-4fd5-a0c6-7ae993177045 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:16.259056 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4902b5d-a310-4fd5-a0c6-7ae993177045 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:16.259328 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4902b5d-a310-4fd5-a0c6-7ae993177045 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:16.272725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4902b5d-a310-4fd5-a0c6-7ae993177045 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:24:16.279204 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-d4902b5d-a310-4fd5-a0c6-7ae993177045 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:24:16.279204 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:16.279204 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:16.279204 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:16.279204 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:16.279204 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:16.279204 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:16.279204 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:16.279204 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:16.279204 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:16.279204 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:16.279204 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:16.279204 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:24:16.279204 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:24:16.279204 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:24:16.279204 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:24:16.280441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:24:16.280441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:24:16.280441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:24:16.280441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:24:16.280441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:16.280441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:16.280441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:16.280441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:16.280441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:16.280441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:16.280441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:16.280441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:16.280441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:16.280441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:16.280441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:16.280441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:16.281350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:16.281350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:16.281350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:16.281350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:16.281350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:24:16.281350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:24:16.281350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:24:16.281350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:24:16.281350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:24:16.281350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:24:16.281350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:24:16.281350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:16.281350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 394/788] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:24:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:24:16.282056 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67279d-887c-4ac3-be5d-3825f0751e74 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:16.282126 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67279d-887c-4ac3-be5d-3825f0751e74 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:16.282372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67279d-887c-4ac3-be5d-3825f0751e74 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:16.284580 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 395/789] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:16.289044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8a873-e1db-4746-a2f5-88342389f9fb None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:16.289587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8a873-e1db-4746-a2f5-88342389f9fb None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:16.289859 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8a873-e1db-4746-a2f5-88342389f9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:16.304736 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca3f060-812f-4b50-9c5d-c979e5106049 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:16.313828 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-65a8a873-e1db-4746-a2f5-88342389f9fb None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:16.315070 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-65a8a873-e1db-4746-a2f5-88342389f9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:16.353440 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa0833-9a7a-4678-b61a-87d51a7ad276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:16.354238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa0833-9a7a-4678-b61a-87d51a7ad276 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:16.354520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa0833-9a7a-4678-b61a-87d51a7ad276 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:16.355688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa0833-9a7a-4678-b61a-87d51a7ad276 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:16.356352 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa0833-9a7a-4678-b61a-87d51a7ad276 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:16.356657 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa0833-9a7a-4678-b61a-87d51a7ad276 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:16.358655 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa0833-9a7a-4678-b61a-87d51a7ad276 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:16.379038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 395/790] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:16.379038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa0833-9a7a-4678-b61a-87d51a7ad276 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:16.402322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 396/791] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:16.403330 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8a873-e1db-4746-a2f5-88342389f9fb None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:16.447021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:16.493618 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:16.495267 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa0833-9a7a-4678-b61a-87d51a7ad276 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:16.527696 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8a873-e1db-4746-a2f5-88342389f9fb None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:16.532428 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73023d5d-93b4-4071-aa21-b2564ccfac44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:16.533327 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73023d5d-93b4-4071-aa21-b2564ccfac44 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:16.533801 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73023d5d-93b4-4071-aa21-b2564ccfac44 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:16.534263 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73023d5d-93b4-4071-aa21-b2564ccfac44 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:16.535126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73023d5d-93b4-4071-aa21-b2564ccfac44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:16.535748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73023d5d-93b4-4071-aa21-b2564ccfac44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:16.537425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73023d5d-93b4-4071-aa21-b2564ccfac44 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:16.547206 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 396/792] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:24:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:16.548281 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73023d5d-93b4-4071-aa21-b2564ccfac44 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:16.566310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38825b-1b67-46c5-b02c-0de5fc393df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:16.567015 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38825b-1b67-46c5-b02c-0de5fc393df2 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:16.567264 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38825b-1b67-46c5-b02c-0de5fc393df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:16.567533 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38825b-1b67-46c5-b02c-0de5fc393df2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:16.575451 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38825b-1b67-46c5-b02c-0de5fc393df2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:16.577129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38825b-1b67-46c5-b02c-0de5fc393df2 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:16.589776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-bb38825b-1b67-46c5-b02c-0de5fc393df2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 08:24:16.589776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:16.589776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:16.589776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:16.589776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:16.589776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:16.589776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:16.589776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:16.589776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:16.589776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:16.589776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:16.589776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:16.589776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:24:16.589776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:24:16.589776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:24:16.589776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:24:16.590724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:16.590724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:16.590724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:24:16.590724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:16.590724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:24:16.590724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:16.590724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:16.590724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:16.590724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:16.590724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:16.590724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:16.590724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:16.590724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:16.590724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:16.590724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:16.590724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:16.590724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:16.590724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:16.591655 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:16.591655 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:16.591655 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:24:16.591655 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:24:16.591655 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:24:16.591655 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:24:16.591655 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:24:16.591655 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:24:16.591655 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 08:24:16.591655 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:16.596360 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 397/793] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:24:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:16.597283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38825b-1b67-46c5-b02c-0de5fc393df2 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:16.661438 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73023d5d-93b4-4071-aa21-b2564ccfac44 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:16.676520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:16.698631 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa66cf3-b316-459d-95ba-a9bcaebc3088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:16.699586 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa66cf3-b316-459d-95ba-a9bcaebc3088 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:16.699968 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa66cf3-b316-459d-95ba-a9bcaebc3088 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:16.700348 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa66cf3-b316-459d-95ba-a9bcaebc3088 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:16.705820 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38825b-1b67-46c5-b02c-0de5fc393df2 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:16.712657 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa66cf3-b316-459d-95ba-a9bcaebc3088 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:24:16.718997 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-caa66cf3-b316-459d-95ba-a9bcaebc3088 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:24:16.718997 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:16.718997 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:16.718997 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:16.718997 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:16.718997 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:16.718997 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:16.718997 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:16.718997 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:16.718997 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:16.718997 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:16.718997 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:16.718997 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:24:16.718997 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:24:16.718997 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:24:16.718997 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:24:16.720095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:24:16.720095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:24:16.720095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:24:16.720095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:24:16.720095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:16.720095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:16.720095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:16.720095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:16.720095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:16.720095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:16.720095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:16.720095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:16.720095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:16.720095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:16.720095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:16.720095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:16.721123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:16.721123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:16.721123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:16.721123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:16.721123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:24:16.721123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:24:16.721123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:24:16.721123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:24:16.721123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:24:16.721123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:24:16.721123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:24:16.721123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:16.723131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 397/794] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 08:24:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:16.723989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa66cf3-b316-459d-95ba-a9bcaebc3088 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:16.742074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:16.748435 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1002adc-a340-4986-aeed-c0c69d1fd9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:16.750188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1002adc-a340-4986-aeed-c0c69d1fd9e4 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:16.750711 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1002adc-a340-4986-aeed-c0c69d1fd9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:16.756848 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1002adc-a340-4986-aeed-c0c69d1fd9e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:16.763445 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1002adc-a340-4986-aeed-c0c69d1fd9e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:16.765415 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1002adc-a340-4986-aeed-c0c69d1fd9e4 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:16.782262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-a1002adc-a340-4986-aeed-c0c69d1fd9e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:16.782262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:16.782262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:16.782262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:16.782262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:16.782262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:16.782262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:16.782262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:16.782262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:16.782262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:16.782262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:16.782262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:16.782262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:16.782262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:16.782262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:16.782262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:16.783349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:16.783349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:16.783349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:16.783349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:16.783349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:16.783349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:16.783349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:16.783349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:16.783349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:16.783349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:16.783349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:16.783349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:16.783349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:16.783349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:16.783349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:16.783349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:16.783349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:16.784268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:16.784268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:16.784268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:16.786334 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 398/795] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:16.787491 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1002adc-a340-4986-aeed-c0c69d1fd9e4 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:16.836680 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa66cf3-b316-459d-95ba-a9bcaebc3088 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:16.874891 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d6d15-bd4b-42bc-9163-b241ba632967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:16.875710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d6d15-bd4b-42bc-9163-b241ba632967 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:16.876089 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d6d15-bd4b-42bc-9163-b241ba632967 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:16.876513 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d6d15-bd4b-42bc-9163-b241ba632967 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:16.877467 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7d6d15-bd4b-42bc-9163-b241ba632967 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:16.877873 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7d6d15-bd4b-42bc-9163-b241ba632967 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:16.879401 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7d6d15-bd4b-42bc-9163-b241ba632967 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:16.903750 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 398/796] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Aug 8 08:24:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:16.903750 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d6d15-bd4b-42bc-9163-b241ba632967 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:16.905670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1002adc-a340-4986-aeed-c0c69d1fd9e4 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:16.932863 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:16.946233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ef4de-85ae-4506-8e9b-23e31d0a2e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:16.946928 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ef4de-85ae-4506-8e9b-23e31d0a2e20 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:16.947267 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ef4de-85ae-4506-8e9b-23e31d0a2e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:16.947545 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ef4de-85ae-4506-8e9b-23e31d0a2e20 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:16.948216 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7ef4de-85ae-4506-8e9b-23e31d0a2e20 None admin] RBAC: Adding query filter params (name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:16.948514 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7ef4de-85ae-4506-8e9b-23e31d0a2e20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:16.949895 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7ef4de-85ae-4506-8e9b-23e31d0a2e20 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:16.959350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 399/797] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 08:24:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:16.962388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ef4de-85ae-4506-8e9b-23e31d0a2e20 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:17.007289 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d6d15-bd4b-42bc-9163-b241ba632967 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:17.016685 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:17.044506 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9af4c-a75d-4fc0-abb8-17857f4a3b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:17.045070 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9af4c-a75d-4fc0-abb8-17857f4a3b35 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:17.045341 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9af4c-a75d-4fc0-abb8-17857f4a3b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:17.045724 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9af4c-a75d-4fc0-abb8-17857f4a3b35 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:17.046387 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9af4c-a75d-4fc0-abb8-17857f4a3b35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:17.046684 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9af4c-a75d-4fc0-abb8-17857f4a3b35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:17.048041 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9af4c-a75d-4fc0-abb8-17857f4a3b35 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:17.059374 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 399/798] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:17.060359 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4115dc8-e473-42a6-a917-eb99f4defc98 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:17.060639 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4115dc8-e473-42a6-a917-eb99f4defc98 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:17.060909 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4115dc8-e473-42a6-a917-eb99f4defc98 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:17.062310 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 400/799] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:17.062851 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4115dc8-e473-42a6-a917-eb99f4defc98 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:17.078978 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ef4de-85ae-4506-8e9b-23e31d0a2e20 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:17.116532 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33839ccd-81f1-4df1-bad4-d9d004570bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:17.117439 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33839ccd-81f1-4df1-bad4-d9d004570bae None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:17.118099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33839ccd-81f1-4df1-bad4-d9d004570bae None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:17.118381 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33839ccd-81f1-4df1-bad4-d9d004570bae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:17.132800 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33839ccd-81f1-4df1-bad4-d9d004570bae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:17.135136 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33839ccd-81f1-4df1-bad4-d9d004570bae None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:17.150199 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-33839ccd-81f1-4df1-bad4-d9d004570bae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:17.150199 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:17.150199 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:17.150199 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:17.150199 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:17.150199 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:17.150199 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:17.150199 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:17.150199 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:17.150199 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:17.150199 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:17.150199 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:17.150199 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:17.150199 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:17.150199 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:17.150199 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:17.151285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:17.151285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:17.151285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:17.151285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:17.151285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:17.151285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:17.151285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:17.151285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:17.151285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:17.151285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:17.151285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:17.151285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:17.151285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:17.151285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:17.151285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:17.151285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:17.151285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:17.152242 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:17.152242 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:17.152242 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:17.153626 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 400/800] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:17.154545 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33839ccd-81f1-4df1-bad4-d9d004570bae None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:17.165638 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:17.176398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4115dc8-e473-42a6-a917-eb99f4defc98 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:17.212729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d0b6c-49de-43b4-ba4e-f843318731c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:17.213345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d0b6c-49de-43b4-ba4e-f843318731c6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:17.213622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d0b6c-49de-43b4-ba4e-f843318731c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:17.213903 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d0b6c-49de-43b4-ba4e-f843318731c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:17.225515 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d0b6c-49de-43b4-ba4e-f843318731c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:17.227119 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d0b6c-49de-43b4-ba4e-f843318731c6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:17.242380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-ca9d0b6c-49de-43b4-ba4e-f843318731c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:17.242380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:17.242380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:17.242380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:17.242380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:17.242380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:17.242380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:17.242380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:17.242380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:17.242380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:17.242380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:17.242380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:17.242380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:17.242380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:17.242380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:17.242380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:17.243376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:17.243376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:17.243376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:17.243376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:17.243376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:17.243376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:17.243376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:17.243376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:17.243376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:17.243376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:17.243376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:17.243376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:17.243376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:17.243376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:17.243376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:17.243376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:17.243376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:17.244264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:17.244264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:17.244264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:17.244264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 401/801] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:17.245740 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9609f-e6c4-420b-8ad1-8c8efc2a39e0 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:17.245740 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9609f-e6c4-420b-8ad1-8c8efc2a39e0 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:17.245740 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9609f-e6c4-420b-8ad1-8c8efc2a39e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:17.259440 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:17.263682 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-00f9609f-e6c4-420b-8ad1-8c8efc2a39e0 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:17.265223 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-00f9609f-e6c4-420b-8ad1-8c8efc2a39e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:17.266096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33839ccd-81f1-4df1-bad4-d9d004570bae None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:17.301236 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a4457-0b63-404b-aaf1-36b0c264f818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:17.301812 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a4457-0b63-404b-aaf1-36b0c264f818 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:17.302117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a4457-0b63-404b-aaf1-36b0c264f818 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:17.302386 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a4457-0b63-404b-aaf1-36b0c264f818 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:17.309818 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a4457-0b63-404b-aaf1-36b0c264f818 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:17.311786 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a4457-0b63-404b-aaf1-36b0c264f818 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:17.322844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-123a4457-0b63-404b-aaf1-36b0c264f818 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 08:24:17.322844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:17.322844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:17.322844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:17.322844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:17.322844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:17.322844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:17.322844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:17.322844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:17.322844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:17.322844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:17.322844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:17.322844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:24:17.322844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:24:17.322844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:24:17.322844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:24:17.324323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:17.324323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:17.324323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:24:17.324323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:17.324323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:24:17.324323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:17.324323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:17.324323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:17.324323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:17.324323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:17.324323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:17.324323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:17.324323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:17.324323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:17.324323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:17.324323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:17.324323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:17.324323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:17.325417 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:17.325417 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:17.325417 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:24:17.325417 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:24:17.325417 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:24:17.325417 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:24:17.325417 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:24:17.325417 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:24:17.325417 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 08:24:17.325417 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:17.325417 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 401/802] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Aug 8 08:24:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:17.325417 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a4457-0b63-404b-aaf1-36b0c264f818 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:17.350279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 402/803] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:17.351965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9609f-e6c4-420b-8ad1-8c8efc2a39e0 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:17.437515 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:17.456741 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a4457-0b63-404b-aaf1-36b0c264f818 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:17.462130 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9609f-e6c4-420b-8ad1-8c8efc2a39e0 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:17.494929 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c82e4-3ca5-4d70-90f5-d5535f262c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:17.495531 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c82e4-3ca5-4d70-90f5-d5535f262c8e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:17.495809 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c82e4-3ca5-4d70-90f5-d5535f262c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:17.496081 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c82e4-3ca5-4d70-90f5-d5535f262c8e None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:17.496736 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671c82e4-3ca5-4d70-90f5-d5535f262c8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:17.497007 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671c82e4-3ca5-4d70-90f5-d5535f262c8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:17.498386 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671c82e4-3ca5-4d70-90f5-d5535f262c8e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:17.499204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981980b-3a9c-408d-9a95-daea1b1bcc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:17.499808 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981980b-3a9c-408d-9a95-daea1b1bcc37 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:17.500092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981980b-3a9c-408d-9a95-daea1b1bcc37 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:17.500369 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981980b-3a9c-408d-9a95-daea1b1bcc37 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:17.500991 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981980b-3a9c-408d-9a95-daea1b1bcc37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:17.501354 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981980b-3a9c-408d-9a95-daea1b1bcc37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:17.502701 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981980b-3a9c-408d-9a95-daea1b1bcc37 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:17.510992 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 402/804] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:17.511611 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c82e4-3ca5-4d70-90f5-d5535f262c8e None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:17.524044 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 403/805] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:17.524044 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981980b-3a9c-408d-9a95-daea1b1bcc37 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:17.531674 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:17.615160 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c82e4-3ca5-4d70-90f5-d5535f262c8e None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:17.627658 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981980b-3a9c-408d-9a95-daea1b1bcc37 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:17.652018 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c14c3a-b5e1-49b8-a121-6780480fa501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:17.652617 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c14c3a-b5e1-49b8-a121-6780480fa501 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:17.652891 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c14c3a-b5e1-49b8-a121-6780480fa501 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:17.653179 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c14c3a-b5e1-49b8-a121-6780480fa501 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:17.653900 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c14c3a-b5e1-49b8-a121-6780480fa501 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:17.654198 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c14c3a-b5e1-49b8-a121-6780480fa501 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:17.655917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c14c3a-b5e1-49b8-a121-6780480fa501 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:17.663629 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138182d8-73a0-4b39-81d5-a2a3871866a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:17.664249 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138182d8-73a0-4b39-81d5-a2a3871866a9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:17.664530 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138182d8-73a0-4b39-81d5-a2a3871866a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:17.664795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138182d8-73a0-4b39-81d5-a2a3871866a9 None admin] PATH_INFO: `/v3/services/network` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:17.665403 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138182d8-73a0-4b39-81d5-a2a3871866a9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:17.666766 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138182d8-73a0-4b39-81d5-a2a3871866a9 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:17.668561 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:17.674204 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 403/806] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Aug 8 08:24:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:17.674851 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c14c3a-b5e1-49b8-a121-6780480fa501 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:17.681589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-138182d8-73a0-4b39-81d5-a2a3871866a9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 08:24:17.681589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:17.681589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 08:24:17.681589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 08:24:17.681589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 08:24:17.681589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 08:24:17.681589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 08:24:17.681589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:17.681589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 08:24:17.681589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:17.681589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:24:17.681589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:17.681589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:17.681589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:17.681589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:17.681589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:17.681589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:17.683469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:17.683469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:17.683469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:17.683469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:17.683469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:17.683469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:17.683469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 08:24:17.683469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 08:24:17.683469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 08:24:17.683469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 08:24:17.683469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:17.683469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:17.683469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:17.683469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:17.683469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:17.683469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:17.684323 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:17.684323 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:17.684323 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:17.684323 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:17.684323 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:17.684323 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:17.684323 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:17.684323 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:17.684323 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:17.684323 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:17.684323 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 08:24:17.684323 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:17.684323 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 08:24:17.684323 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:17.687557 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 404/807] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:24:17 2022] GET /identity/v3/services/network => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:17.688403 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138182d8-73a0-4b39-81d5-a2a3871866a9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:17.752726 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:17.776673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c14c3a-b5e1-49b8-a121-6780480fa501 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:17.788969 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138182d8-73a0-4b39-81d5-a2a3871866a9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:17.810557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b008804-6e81-44c1-8dce-d8e562815268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:17.811192 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b008804-6e81-44c1-8dce-d8e562815268 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:17.811478 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b008804-6e81-44c1-8dce-d8e562815268 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:17.811744 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b008804-6e81-44c1-8dce-d8e562815268 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:17.818423 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b008804-6e81-44c1-8dce-d8e562815268 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:17.819840 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b008804-6e81-44c1-8dce-d8e562815268 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:17.821845 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f2efc-93fc-4b79-9b31-29b82549e18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:17.822463 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f2efc-93fc-4b79-9b31-29b82549e18b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:17.822738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f2efc-93fc-4b79-9b31-29b82549e18b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:17.823008 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f2efc-93fc-4b79-9b31-29b82549e18b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:17.831018 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-7b008804-6e81-44c1-8dce-d8e562815268 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:17.831018 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:17.831018 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:17.831018 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:17.831018 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:17.831018 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:17.831018 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:17.831018 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:17.831018 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:17.831018 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:17.831018 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:17.831018 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:17.831018 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:24:17.831018 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:24:17.831018 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:24:17.832032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:24:17.832032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:17.832032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:17.832032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:17.832032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:17.832032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:17.832032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:17.832032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:17.832032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:17.832032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:17.832032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:17.832032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:17.832032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:17.832032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:17.832032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:17.832032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:17.832032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:17.833923 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:24:17.833923 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:24:17.833923 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:24:17.833923 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:24:17.833923 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:24:17.833923 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:24:17.833923 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:17.833923 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:17.835055 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f2efc-93fc-4b79-9b31-29b82549e18b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:17.835207 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 404/808] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:24:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:17.835207 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b008804-6e81-44c1-8dce-d8e562815268 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:17.835555 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f2efc-93fc-4b79-9b31-29b82549e18b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:17.845330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-627f2efc-93fc-4b79-9b31-29b82549e18b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 08:24:17.845330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:17.845330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:17.845330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:17.845330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:17.845330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:17.845330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:17.845330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:17.845330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:17.845330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:17.845330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:17.845330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:17.845330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:24:17.845330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:24:17.845330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:24:17.845330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:24:17.846900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:17.846900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:17.846900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:24:17.846900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:17.846900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:24:17.846900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:17.846900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:17.846900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:17.846900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:17.846900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:17.846900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:17.846900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:17.846900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:17.846900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:17.846900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:17.846900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:17.846900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:17.846900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:17.848228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:17.848228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:17.848228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:24:17.848228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:24:17.848228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:24:17.848228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:24:17.848228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:24:17.848228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:24:17.848228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 08:24:17.848228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:17.848228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 405/809] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Aug 8 08:24:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:17.848228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f2efc-93fc-4b79-9b31-29b82549e18b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:17.905317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:17.932438 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b008804-6e81-44c1-8dce-d8e562815268 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:17.946324 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f2efc-93fc-4b79-9b31-29b82549e18b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:17.965679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6335b-8625-4239-97a5-00a497c59c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:17.966280 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6335b-8625-4239-97a5-00a497c59c29 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:17.966556 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6335b-8625-4239-97a5-00a497c59c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:17.966827 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6335b-8625-4239-97a5-00a497c59c29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:17.969593 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:17.979852 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef6e479-e9c8-4090-b723-49cb3dabd644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:17.980589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6e479-e9c8-4090-b723-49cb3dabd644 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:17.982129 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6e479-e9c8-4090-b723-49cb3dabd644 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:17.982129 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6e479-e9c8-4090-b723-49cb3dabd644 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:17.982129 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef6e479-e9c8-4090-b723-49cb3dabd644 None admin] RBAC: Adding query filter params (name=network) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:17.982129 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef6e479-e9c8-4090-b723-49cb3dabd644 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:17.983970 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef6e479-e9c8-4090-b723-49cb3dabd644 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:17.988350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 406/810] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Aug 8 08:24:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:17.989526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c43e85-724a-42d0-bf6a-30e87771dd5a None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:17.989811 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c43e85-724a-42d0-bf6a-30e87771dd5a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:17.990086 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c43e85-724a-42d0-bf6a-30e87771dd5a None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:17.995145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 407/811] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:17.995145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c43e85-724a-42d0-bf6a-30e87771dd5a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:18.040102 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6335b-8625-4239-97a5-00a497c59c29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:18.041609 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6335b-8625-4239-97a5-00a497c59c29 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:18.093664 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c43e85-724a-42d0-bf6a-30e87771dd5a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:18.115268 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:18.126468 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5a77d-936c-4f68-ad5e-a79f27056de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:18.127061 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5a77d-936c-4f68-ad5e-a79f27056de0 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:18.127338 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5a77d-936c-4f68-ad5e-a79f27056de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:18.127607 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5a77d-936c-4f68-ad5e-a79f27056de0 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:18.128286 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5a77d-936c-4f68-ad5e-a79f27056de0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:18.128591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5a77d-936c-4f68-ad5e-a79f27056de0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:18.130144 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5a77d-936c-4f68-ad5e-a79f27056de0 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:18.138675 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 405/812] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:24:17 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 08:24:18.139380 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6335b-8625-4239-97a5-00a497c59c29 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:18.142339 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 408/813] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:24:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:18.144459 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5a77d-936c-4f68-ad5e-a79f27056de0 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:18.172675 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:18.238598 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6335b-8625-4239-97a5-00a497c59c29 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:18.242294 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5a77d-936c-4f68-ad5e-a79f27056de0 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:18.272407 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6daaf9-73c0-420d-8c99-6adefe11f108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:18.273013 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6daaf9-73c0-420d-8c99-6adefe11f108 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:18.273332 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6daaf9-73c0-420d-8c99-6adefe11f108 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:18.273605 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6daaf9-73c0-420d-8c99-6adefe11f108 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:18.274327 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6daaf9-73c0-420d-8c99-6adefe11f108 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:18.274629 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6daaf9-73c0-420d-8c99-6adefe11f108 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:18.275580 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7619d-5f38-49cd-b453-23b4a9719d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:18.276033 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6daaf9-73c0-420d-8c99-6adefe11f108 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:18.276176 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7619d-5f38-49cd-b453-23b4a9719d48 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:18.276452 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7619d-5f38-49cd-b453-23b4a9719d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:18.276719 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7619d-5f38-49cd-b453-23b4a9719d48 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:18.277368 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c7619d-5f38-49cd-b453-23b4a9719d48 None admin] RBAC: Adding query filter params (type=network) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:18.277667 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c7619d-5f38-49cd-b453-23b4a9719d48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:18.279020 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c7619d-5f38-49cd-b453-23b4a9719d48 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:18.284121 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 409/814] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Aug 8 08:24:18 2022] GET /identity/v3/services?type=network => generated 350 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:18.285275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3dde5-c614-4e90-86ad-4c4d7c570460 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:18.285556 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3dde5-c614-4e90-86ad-4c4d7c570460 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:18.285822 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3dde5-c614-4e90-86ad-4c4d7c570460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:18.294435 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 406/815] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Aug 8 08:24:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:18.295392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6daaf9-73c0-420d-8c99-6adefe11f108 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:18.303472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c4a3dde5-c614-4e90-86ad-4c4d7c570460 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:18.304649 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c4a3dde5-c614-4e90-86ad-4c4d7c570460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:18.346230 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:18.369648 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 410/816] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:18.370447 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a3dde5-c614-4e90-86ad-4c4d7c570460 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:18.389278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:18.398044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6daaf9-73c0-420d-8c99-6adefe11f108 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:18.432251 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df51dc-e709-4970-a877-cb12af82c3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:18.432795 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df51dc-e709-4970-a877-cb12af82c3be None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:18.433072 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df51dc-e709-4970-a877-cb12af82c3be None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:18.433366 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df51dc-e709-4970-a877-cb12af82c3be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:18.444396 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df51dc-e709-4970-a877-cb12af82c3be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:18.445855 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df51dc-e709-4970-a877-cb12af82c3be None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:18.460221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-89df51dc-e709-4970-a877-cb12af82c3be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:18.460221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:18.460221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:18.460221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:18.460221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:18.460221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:18.460221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:18.460221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:18.460221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:18.460221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:18.460221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:18.460221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:18.460221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:18.460221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:18.460221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:18.460221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:18.461308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:18.461308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:18.461308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:18.461308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:18.461308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:18.461308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:18.461308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:18.461308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:18.461308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:18.461308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:18.461308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:18.461308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:18.461308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:18.461308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:18.461308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:18.461308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:18.461308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:18.462398 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:18.462398 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:18.462398 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:18.462398 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 407/817] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:18.462398 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df51dc-e709-4970-a877-cb12af82c3be None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:18.474449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a3dde5-c614-4e90-86ad-4c4d7c570460 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:18.507855 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123a732-aed2-45c8-8945-db49965cc8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:18.508507 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123a732-aed2-45c8-8945-db49965cc8b9 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:18.508784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123a732-aed2-45c8-8945-db49965cc8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:18.509056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123a732-aed2-45c8-8945-db49965cc8b9 None admin] PATH_INFO: `/v3/projects/71e5e82c9f454cdd91dfeac5a57968c4/users/e69c0e02ad944f2a85fc5318018e5cb0/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:18.510958 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123a732-aed2-45c8-8945-db49965cc8b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=71e5e82c9f454cdd91dfeac5a57968c4, user_id=e69c0e02ad944f2a85fc5318018e5cb0, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:18.512382 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123a732-aed2-45c8-8945-db49965cc8b9 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:18.559234 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df51dc-e709-4970-a877-cb12af82c3be None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:18.591405 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6403f81-7baf-4fe8-b36b-975b95a9d4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:18.591998 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6403f81-7baf-4fe8-b36b-975b95a9d4d5 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:18.592273 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6403f81-7baf-4fe8-b36b-975b95a9d4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:18.592676 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6403f81-7baf-4fe8-b36b-975b95a9d4d5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:18.593193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 411/818] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 08:24:18 2022] PUT /identity/v3/projects/71e5e82c9f454cdd91dfeac5a57968c4/users/e69c0e02ad944f2a85fc5318018e5cb0/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:24:18.605305 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6403f81-7baf-4fe8-b36b-975b95a9d4d5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:18.605511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123a732-aed2-45c8-8945-db49965cc8b9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:18.605608 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6403f81-7baf-4fe8-b36b-975b95a9d4d5 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:18.633919 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:18.698414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:18.702313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123a732-aed2-45c8-8945-db49965cc8b9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:18.709791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 408/819] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:24:18 2022] POST /identity/v3/endpoints => generated 337 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:24:18.714358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6403f81-7baf-4fe8-b36b-975b95a9d4d5 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:18.763155 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712f119-840a-4913-bfc1-9a8ea461d49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:18.765490 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712f119-840a-4913-bfc1-9a8ea461d49a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:18.765490 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712f119-840a-4913-bfc1-9a8ea461d49a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:18.765490 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712f119-840a-4913-bfc1-9a8ea461d49a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:18.778631 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5712f119-840a-4913-bfc1-9a8ea461d49a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:18.780646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5712f119-840a-4913-bfc1-9a8ea461d49a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:18.801996 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-5712f119-840a-4913-bfc1-9a8ea461d49a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:18.801996 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:18.801996 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:18.801996 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:18.801996 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:18.801996 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:18.801996 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:18.801996 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:18.801996 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:18.801996 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:18.801996 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:18.801996 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:18.801996 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:18.801996 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:18.801996 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:18.801996 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:18.803199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:18.803199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:18.803199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:18.803199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:18.803199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:18.803199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:18.803199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:18.803199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:18.803199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:18.803199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:18.803199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:18.803199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:18.803199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:18.803199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:18.803199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:18.803199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:18.803199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:18.804178 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:18.804178 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:18.804178 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:18.804178 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 412/820] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:18.804503 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712f119-840a-4913-bfc1-9a8ea461d49a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:18.875632 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6403f81-7baf-4fe8-b36b-975b95a9d4d5 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:18.920485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:18.929937 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ac76b-7def-4b0e-912c-d8f998b3ea00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:18.930850 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ac76b-7def-4b0e-912c-d8f998b3ea00 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:18.931553 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ac76b-7def-4b0e-912c-d8f998b3ea00 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:18.931990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ac76b-7def-4b0e-912c-d8f998b3ea00 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:18.932903 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735ac76b-7def-4b0e-912c-d8f998b3ea00 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:18.935022 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735ac76b-7def-4b0e-912c-d8f998b3ea00 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:18.939758 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712f119-840a-4913-bfc1-9a8ea461d49a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:18.945625 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-735ac76b-7def-4b0e-912c-d8f998b3ea00 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 08:24:18.945625 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:18.945625 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 08:24:18.945625 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 08:24:18.945625 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 08:24:18.945625 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 08:24:18.945625 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 08:24:18.945625 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:18.945625 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 08:24:18.945625 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:18.945625 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:24:18.945625 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:18.945625 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:18.945625 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:18.945625 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:18.945625 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:18.945625 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:18.947278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:18.947278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:18.947278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:18.947278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:18.947278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:18.947278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:18.947278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 08:24:18.947278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 08:24:18.947278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 08:24:18.947278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 08:24:18.947278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:18.947278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:18.947278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:18.947278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:18.947278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:18.947278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:18.948323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:18.948323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:18.948323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:18.948323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:18.948323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:18.948323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:18.948323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:18.948323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:18.948323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:18.948323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:18.948323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 08:24:18.948323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:18.948323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 08:24:18.948323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:18.953524 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 409/821] 10.222.0.33 () {56 vars in 1129 bytes} [Mon Aug 8 08:24:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:18.953524 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ac76b-7def-4b0e-912c-d8f998b3ea00 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:18.984838 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31311386-089c-46f6-8dba-47151de48eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:18.985656 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31311386-089c-46f6-8dba-47151de48eeb None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:18.985937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31311386-089c-46f6-8dba-47151de48eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:18.986204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31311386-089c-46f6-8dba-47151de48eeb None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:18.986845 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31311386-089c-46f6-8dba-47151de48eeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:18.987200 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31311386-089c-46f6-8dba-47151de48eeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:18.988571 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31311386-089c-46f6-8dba-47151de48eeb None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:19.005185 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 413/822] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:19.005185 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31311386-089c-46f6-8dba-47151de48eeb None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:19.027967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:19.068221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ac76b-7def-4b0e-912c-d8f998b3ea00 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:19.105119 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017104c2-98d6-494a-b320-fe3739ee60f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:19.105774 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017104c2-98d6-494a-b320-fe3739ee60f0 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:19.106048 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017104c2-98d6-494a-b320-fe3739ee60f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:19.106322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017104c2-98d6-494a-b320-fe3739ee60f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:19.106960 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017104c2-98d6-494a-b320-fe3739ee60f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:19.107257 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017104c2-98d6-494a-b320-fe3739ee60f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:19.109302 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017104c2-98d6-494a-b320-fe3739ee60f0 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:19.115279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31311386-089c-46f6-8dba-47151de48eeb None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:19.126190 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 410/823] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:19.126190 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017104c2-98d6-494a-b320-fe3739ee60f0 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:19.157755 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf13db-1b2c-4351-ae3e-0fe8fe4417e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:19.158677 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf13db-1b2c-4351-ae3e-0fe8fe4417e4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:19.159208 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf13db-1b2c-4351-ae3e-0fe8fe4417e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:19.159482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf13db-1b2c-4351-ae3e-0fe8fe4417e4 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:19.160111 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf13db-1b2c-4351-ae3e-0fe8fe4417e4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:19.160406 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf13db-1b2c-4351-ae3e-0fe8fe4417e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:19.162707 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf13db-1b2c-4351-ae3e-0fe8fe4417e4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:19.169303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 414/824] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Aug 8 08:24:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:19.170114 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf13db-1b2c-4351-ae3e-0fe8fe4417e4 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:19.182326 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:19.234490 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017104c2-98d6-494a-b320-fe3739ee60f0 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:19.259991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:19.271233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ea0b8-ff25-4429-91c2-f88725c343a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:19.274524 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ea0b8-ff25-4429-91c2-f88725c343a0 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:19.274524 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ea0b8-ff25-4429-91c2-f88725c343a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:19.274524 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ea0b8-ff25-4429-91c2-f88725c343a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:19.279175 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ea0b8-ff25-4429-91c2-f88725c343a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:19.280525 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf13db-1b2c-4351-ae3e-0fe8fe4417e4 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:19.280767 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ea0b8-ff25-4429-91c2-f88725c343a0 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:19.292063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-6d7ea0b8-ff25-4429-91c2-f88725c343a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:19.292063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:19.292063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:19.292063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:19.292063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:19.292063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:19.292063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:19.292063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:19.292063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:19.292063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:19.292063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:19.292063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:19.292063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:24:19.292063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:24:19.292063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:24:19.293131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:24:19.293131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:19.293131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:19.293131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:19.293131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:19.293131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:19.293131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:19.293131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:19.293131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:19.293131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:19.293131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:19.293131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:19.293131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:19.293131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:19.293131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:19.293131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:19.293131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:19.294286 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:24:19.294286 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:24:19.294286 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:24:19.294286 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:24:19.294286 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:24:19.294286 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:24:19.294286 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:19.294286 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:19.301299 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 411/825] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:24:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:19.302438 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ea0b8-ff25-4429-91c2-f88725c343a0 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:19.319918 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554d614-c914-4f03-825d-098fc7482832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:19.320547 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554d614-c914-4f03-825d-098fc7482832 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:19.320955 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554d614-c914-4f03-825d-098fc7482832 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:19.321124 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554d614-c914-4f03-825d-098fc7482832 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:19.328523 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a554d614-c914-4f03-825d-098fc7482832 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:19.329998 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a554d614-c914-4f03-825d-098fc7482832 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:19.342512 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-a554d614-c914-4f03-825d-098fc7482832 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:19.342512 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:19.342512 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:19.342512 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:19.342512 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:19.342512 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:19.342512 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:19.342512 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:19.342512 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:19.342512 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:19.342512 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:19.342512 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:19.342512 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:24:19.342512 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:24:19.342512 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:24:19.343395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:24:19.343395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:19.343395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:19.343395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:19.343395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:19.343395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:19.343395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:19.343395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:19.343395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:19.343395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:19.343395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:19.343395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:19.343395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:19.343395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:19.343395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:19.343395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:19.343395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:19.344276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:24:19.344276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:24:19.344276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:24:19.344276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:24:19.344276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:24:19.344276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:24:19.344276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:19.344276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:19.348017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 415/826] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:24:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:19.348965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554d614-c914-4f03-825d-098fc7482832 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:19.419215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:19.426175 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ea0b8-ff25-4429-91c2-f88725c343a0 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:19.453909 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554d614-c914-4f03-825d-098fc7482832 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:19.462878 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f011f9-0334-439e-9811-a2b2e679234d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:19.463485 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f011f9-0334-439e-9811-a2b2e679234d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:19.463766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f011f9-0334-439e-9811-a2b2e679234d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:19.464039 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f011f9-0334-439e-9811-a2b2e679234d None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:19.464781 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f011f9-0334-439e-9811-a2b2e679234d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:19.465082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f011f9-0334-439e-9811-a2b2e679234d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:19.466476 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f011f9-0334-439e-9811-a2b2e679234d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:19.472657 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 412/827] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Aug 8 08:24:19 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:19.473333 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f011f9-0334-439e-9811-a2b2e679234d None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:19.493824 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b7525-159b-4c6b-ae5a-1633d4b83941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:19.494363 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b7525-159b-4c6b-ae5a-1633d4b83941 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:19.494644 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b7525-159b-4c6b-ae5a-1633d4b83941 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:19.494918 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b7525-159b-4c6b-ae5a-1633d4b83941 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:19.495605 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b7525-159b-4c6b-ae5a-1633d4b83941 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:19.495907 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b7525-159b-4c6b-ae5a-1633d4b83941 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:19.497357 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b7525-159b-4c6b-ae5a-1633d4b83941 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:19.514625 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 416/828] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:24:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:19.514625 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b7525-159b-4c6b-ae5a-1633d4b83941 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:19.532522 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:19.589636 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f011f9-0334-439e-9811-a2b2e679234d None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:19.614579 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b7525-159b-4c6b-ae5a-1633d4b83941 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:19.627008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db88b3b-02a2-4c34-9530-42395131f646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:19.627618 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db88b3b-02a2-4c34-9530-42395131f646 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:19.627898 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db88b3b-02a2-4c34-9530-42395131f646 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:19.628168 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db88b3b-02a2-4c34-9530-42395131f646 None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:19.628857 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db88b3b-02a2-4c34-9530-42395131f646 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:19.629180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db88b3b-02a2-4c34-9530-42395131f646 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:19.630637 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db88b3b-02a2-4c34-9530-42395131f646 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:19.644768 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 413/829] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:24:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:19.645625 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db88b3b-02a2-4c34-9530-42395131f646 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:19.651856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:19.660704 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c366dd-2f6a-46a4-b787-65fba0065758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:19.666533 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c366dd-2f6a-46a4-b787-65fba0065758 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:19.666533 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c366dd-2f6a-46a4-b787-65fba0065758 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:19.666533 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c366dd-2f6a-46a4-b787-65fba0065758 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:19.666533 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c366dd-2f6a-46a4-b787-65fba0065758 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:19.666533 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c366dd-2f6a-46a4-b787-65fba0065758 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:19.733747 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 417/830] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:24:19 2022] POST /identity/v3/endpoints => generated 346 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:24:19.733747 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c366dd-2f6a-46a4-b787-65fba0065758 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:19.754155 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db88b3b-02a2-4c34-9530-42395131f646 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:19.765823 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:19.801194 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239aa71-f685-44d5-a081-d77316943342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:19.801821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239aa71-f685-44d5-a081-d77316943342 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:19.802095 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239aa71-f685-44d5-a081-d77316943342 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:19.802416 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239aa71-f685-44d5-a081-d77316943342 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:19.803203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239aa71-f685-44d5-a081-d77316943342 None admin] RBAC: Adding query filter params (role.id=c1e514a4b5424f959b069f36cbfd8147, scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4, user.id=a3317f751ec74bde95ec262394001292) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:19.803513 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239aa71-f685-44d5-a081-d77316943342 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:19.805299 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239aa71-f685-44d5-a081-d77316943342 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:19.821372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 414/831] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:24:19 2022] GET /identity/v3/role_assignments?user.id=a3317f751ec74bde95ec262394001292&scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4&role.id=c1e514a4b5424f959b069f36cbfd8147 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:19.872499 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c366dd-2f6a-46a4-b787-65fba0065758 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:19.915868 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debeabf9-1095-4026-b915-989d152e7c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:19.916912 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debeabf9-1095-4026-b915-989d152e7c42 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:19.917376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debeabf9-1095-4026-b915-989d152e7c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:19.917803 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debeabf9-1095-4026-b915-989d152e7c42 None admin] PATH_INFO: `/v3/projects/71e5e82c9f454cdd91dfeac5a57968c4/users/7f3422c6f9a041ceba851b71bd15d0df/roles/402c3e6fe3364285bd72784a297db11e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:19.937333 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debeabf9-1095-4026-b915-989d152e7c42 None admin] RBAC: Authorizing `identity:create_grant(project_id=71e5e82c9f454cdd91dfeac5a57968c4, user_id=7f3422c6f9a041ceba851b71bd15d0df, role_id=402c3e6fe3364285bd72784a297db11e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:19.938804 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debeabf9-1095-4026-b915-989d152e7c42 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:20.017507 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 418/832] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 08:24:19 2022] PUT /identity/v3/projects/71e5e82c9f454cdd91dfeac5a57968c4/users/7f3422c6f9a041ceba851b71bd15d0df/roles/402c3e6fe3364285bd72784a297db11e => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:24:20.660337 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b178e-13a7-4607-9a0d-4d69d962cb9a None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:20.660854 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b178e-13a7-4607-9a0d-4d69d962cb9a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:20.660854 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b178e-13a7-4607-9a0d-4d69d962cb9a None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:20.662349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 415/833] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:20.667776 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d892a4a-8c1e-4b0c-99df-f0041f9c6a14 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:20.668557 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d892a4a-8c1e-4b0c-99df-f0041f9c6a14 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:20.669029 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d892a4a-8c1e-4b0c-99df-f0041f9c6a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:20.689475 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4d892a4a-8c1e-4b0c-99df-f0041f9c6a14 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:20.690976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4d892a4a-8c1e-4b0c-99df-f0041f9c6a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:20.717343 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:20.757923 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 419/834] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:20 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 08:24:20.781716 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b178e-13a7-4607-9a0d-4d69d962cb9a None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:20.893135 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b178e-13a7-4607-9a0d-4d69d962cb9a None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:20.929353 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b8aca-93a1-4c48-ad43-fcc6e3b3e43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:20.930337 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b8aca-93a1-4c48-ad43-fcc6e3b3e43f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:20.930811 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b8aca-93a1-4c48-ad43-fcc6e3b3e43f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:20.931330 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b8aca-93a1-4c48-ad43-fcc6e3b3e43f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:20.943833 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b8aca-93a1-4c48-ad43-fcc6e3b3e43f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:24:20.949240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-b24b8aca-93a1-4c48-ad43-fcc6e3b3e43f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:24:20.949240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:20.949240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:20.949240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:20.949240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:20.949240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:20.949240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:20.949240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:20.949240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:20.949240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:20.949240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:20.949240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:20.949240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:24:20.949240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:24:20.949240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:24:20.949240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:24:20.950217 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:24:20.950217 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:24:20.950217 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:24:20.950217 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:24:20.950217 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:20.950217 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:20.950217 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:20.950217 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:20.950217 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:20.950217 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:20.950217 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:20.950217 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:20.950217 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:20.950217 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:20.950217 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:20.950217 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:20.951042 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:20.951042 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:20.951042 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:20.951042 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:20.951042 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:24:20.951042 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:24:20.951042 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:24:20.951042 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:24:20.951042 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:24:20.951042 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:24:20.951042 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:24:20.951042 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:20.955014 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 416/835] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:24:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:20.961674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d892a4a-8c1e-4b0c-99df-f0041f9c6a14 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:21.072283 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d892a4a-8c1e-4b0c-99df-f0041f9c6a14 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:21.108932 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d6d4f-6afe-4075-94b0-4a0d1b637030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:21.109586 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d6d4f-6afe-4075-94b0-4a0d1b637030 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:21.109865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d6d4f-6afe-4075-94b0-4a0d1b637030 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:21.110130 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d6d4f-6afe-4075-94b0-4a0d1b637030 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:21.110795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d6d4f-6afe-4075-94b0-4a0d1b637030 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:21.111091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d6d4f-6afe-4075-94b0-4a0d1b637030 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:21.112453 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d6d4f-6afe-4075-94b0-4a0d1b637030 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:21.121001 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 420/836] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:24:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:21.127119 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b8aca-93a1-4c48-ad43-fcc6e3b3e43f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:21.157925 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:21.235636 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b8aca-93a1-4c48-ad43-fcc6e3b3e43f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:21.271560 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df86561-cda2-4375-bc2d-2318ccbb54b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:21.272411 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df86561-cda2-4375-bc2d-2318ccbb54b0 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:21.272826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df86561-cda2-4375-bc2d-2318ccbb54b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:21.273261 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df86561-cda2-4375-bc2d-2318ccbb54b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:21.284156 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df86561-cda2-4375-bc2d-2318ccbb54b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:21.285784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df86561-cda2-4375-bc2d-2318ccbb54b0 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:21.300315 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-6df86561-cda2-4375-bc2d-2318ccbb54b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:21.300315 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:21.300315 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:21.300315 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:21.300315 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:21.300315 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:21.300315 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:21.300315 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:21.300315 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:21.300315 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:21.300315 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:21.300315 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:21.300315 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:21.300315 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:21.300315 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:21.300315 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:21.301469 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:21.301469 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:21.301469 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:21.301469 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:21.301469 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:21.301469 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:21.301469 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:21.301469 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:21.301469 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:21.301469 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:21.301469 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:21.301469 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:21.301469 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:21.301469 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:21.301469 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:21.301469 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:21.301469 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:21.302527 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:21.302527 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:21.302527 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:21.304275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 417/837] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:21.309714 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d6d4f-6afe-4075-94b0-4a0d1b637030 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:21.338357 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:21.420274 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d6d4f-6afe-4075-94b0-4a0d1b637030 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:21.459238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825ca20-2f52-4bbf-9876-d0c99f6627b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:21.459848 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825ca20-2f52-4bbf-9876-d0c99f6627b5 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:21.460122 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825ca20-2f52-4bbf-9876-d0c99f6627b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:21.460392 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825ca20-2f52-4bbf-9876-d0c99f6627b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:21.461092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825ca20-2f52-4bbf-9876-d0c99f6627b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:21.461426 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825ca20-2f52-4bbf-9876-d0c99f6627b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:21.462874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825ca20-2f52-4bbf-9876-d0c99f6627b5 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:21.475465 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 421/838] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:21.480979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df86561-cda2-4375-bc2d-2318ccbb54b0 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:21.569546 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:21.589339 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df86561-cda2-4375-bc2d-2318ccbb54b0 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:21.624451 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc1972-f444-4767-9d81-69dc3e09defb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:21.625072 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc1972-f444-4767-9d81-69dc3e09defb None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:21.625375 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc1972-f444-4767-9d81-69dc3e09defb None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:21.625643 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc1972-f444-4767-9d81-69dc3e09defb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:21.633254 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecc1972-f444-4767-9d81-69dc3e09defb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:21.635960 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecc1972-f444-4767-9d81-69dc3e09defb None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:21.647780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-aecc1972-f444-4767-9d81-69dc3e09defb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 08:24:21.647780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:21.647780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:21.647780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:21.647780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:21.647780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:21.647780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:21.647780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:21.647780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:21.647780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:21.647780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:21.647780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:21.647780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:24:21.647780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:24:21.647780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:24:21.647780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:24:21.648720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:21.648720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:21.648720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:24:21.648720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:21.648720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:24:21.648720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:21.648720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:21.648720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:21.648720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:21.648720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:21.648720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:21.648720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:21.648720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:21.648720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:21.648720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:21.648720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:21.648720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:21.648720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:21.651414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:21.651414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:21.651414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:24:21.651414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:24:21.651414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:24:21.651414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:24:21.651414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:24:21.651414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:24:21.651414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 08:24:21.651414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:21.651414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 418/839] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:24:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:21.654664 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825ca20-2f52-4bbf-9876-d0c99f6627b5 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:21.754714 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:21.769820 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825ca20-2f52-4bbf-9876-d0c99f6627b5 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:21.805930 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d6ca0d-cd19-4493-a4df-ab75ee1d1d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:21.806576 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6ca0d-cd19-4493-a4df-ab75ee1d1d40 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:21.806861 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6ca0d-cd19-4493-a4df-ab75ee1d1d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:21.807129 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6ca0d-cd19-4493-a4df-ab75ee1d1d40 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:21.807856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d6ca0d-cd19-4493-a4df-ab75ee1d1d40 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:21.808157 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d6ca0d-cd19-4493-a4df-ab75ee1d1d40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:21.809776 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d6ca0d-cd19-4493-a4df-ab75ee1d1d40 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:21.828645 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 422/840] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Aug 8 08:24:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:21.834596 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc1972-f444-4767-9d81-69dc3e09defb None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:21.914717 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345eb20-cbc0-4dbd-9dba-d80d12e15aa3 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:21.915000 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345eb20-cbc0-4dbd-9dba-d80d12e15aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:21.915277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345eb20-cbc0-4dbd-9dba-d80d12e15aa3 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:21.916771 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 423/841] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:21.922796 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60471dee-27af-4d6e-a44b-3b030df1526c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:21.923237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60471dee-27af-4d6e-a44b-3b030df1526c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:21.923573 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60471dee-27af-4d6e-a44b-3b030df1526c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:21.941611 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc1972-f444-4767-9d81-69dc3e09defb None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:21.942385 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-60471dee-27af-4d6e-a44b-3b030df1526c None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:21.951507 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-60471dee-27af-4d6e-a44b-3b030df1526c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:21.976739 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d1f522-ceee-47fe-82d0-88973882db6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:21.977613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1f522-ceee-47fe-82d0-88973882db6c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:21.977890 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1f522-ceee-47fe-82d0-88973882db6c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:21.978160 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1f522-ceee-47fe-82d0-88973882db6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:21.989881 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:22.020580 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 424/842] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:21 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 08:24:22.022628 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11535d68-8a9b-423c-a139-aec019fc3b08 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:22.023338 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11535d68-8a9b-423c-a139-aec019fc3b08 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:22.023803 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11535d68-8a9b-423c-a139-aec019fc3b08 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:22.025981 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 425/843] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:22.031537 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4da775-64eb-4a6f-ac64-3a6a82777dfa None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:22.032054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4da775-64eb-4a6f-ac64-3a6a82777dfa None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:22.032643 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4da775-64eb-4a6f-ac64-3a6a82777dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:22.053801 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d1f522-ceee-47fe-82d0-88973882db6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:22.054785 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8f4da775-64eb-4a6f-ac64-3a6a82777dfa None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:22.055357 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d1f522-ceee-47fe-82d0-88973882db6c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:22.055949 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8f4da775-64eb-4a6f-ac64-3a6a82777dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:22.137171 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 426/844] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:22 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:22.137171 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4da775-64eb-4a6f-ac64-3a6a82777dfa None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:22.141432 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:22.171880 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 419/845] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:24:21 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 08:24:22.172668 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d1f522-ceee-47fe-82d0-88973882db6c None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:22.213130 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:22.239199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4da775-64eb-4a6f-ac64-3a6a82777dfa None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:22.269069 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d1f522-ceee-47fe-82d0-88973882db6c None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:22.273119 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23a271-35e5-4422-84e9-fc85fad7eca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:22.273775 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23a271-35e5-4422-84e9-fc85fad7eca3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:22.274050 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23a271-35e5-4422-84e9-fc85fad7eca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:22.274318 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23a271-35e5-4422-84e9-fc85fad7eca3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:22.274890 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23a271-35e5-4422-84e9-fc85fad7eca3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:22.276260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23a271-35e5-4422-84e9-fc85fad7eca3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:22.285963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-8a23a271-35e5-4422-84e9-fc85fad7eca3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 08:24:22.285963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:22.285963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 08:24:22.285963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 08:24:22.285963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 08:24:22.285963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 08:24:22.285963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 08:24:22.285963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:22.285963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 08:24:22.285963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:22.285963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:24:22.285963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:22.285963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:22.285963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:22.285963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:22.285963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:22.287114 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:22.287114 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:22.287114 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:22.287114 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:22.287114 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:22.287114 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:22.287114 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:22.287114 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 08:24:22.287114 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 08:24:22.287114 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 08:24:22.287114 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 08:24:22.287114 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:22.287114 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:22.287114 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:22.287114 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:22.287114 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:22.287114 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:22.288201 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:22.288201 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:22.288201 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:22.288201 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:22.288201 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:22.288201 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:22.288201 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:22.288201 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:22.288201 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:22.288201 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:22.288201 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 08:24:22.288201 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:22.288201 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 08:24:22.288201 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:22.288201 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 427/846] 10.222.0.33 () {56 vars in 1132 bytes} [Mon Aug 8 08:24:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:22.289180 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23a271-35e5-4422-84e9-fc85fad7eca3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:22.306325 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a48a6-acad-4141-9422-e253360e3b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:22.306942 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a48a6-acad-4141-9422-e253360e3b04 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:22.307215 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a48a6-acad-4141-9422-e253360e3b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:22.307478 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a48a6-acad-4141-9422-e253360e3b04 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:22.308048 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a48a6-acad-4141-9422-e253360e3b04 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:22.309409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a48a6-acad-4141-9422-e253360e3b04 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:22.317641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-ea2a48a6-acad-4141-9422-e253360e3b04 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 08:24:22.317641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:22.317641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 08:24:22.317641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 08:24:22.317641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 08:24:22.317641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 08:24:22.317641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 08:24:22.317641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:22.317641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 08:24:22.317641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:22.317641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:24:22.317641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:22.317641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:22.317641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:22.317641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:22.317641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:22.317641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:22.318619 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:22.318619 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:22.318619 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:22.318619 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:22.318619 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:22.318619 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:22.318619 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 08:24:22.318619 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 08:24:22.318619 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 08:24:22.318619 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 08:24:22.318619 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:22.318619 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:22.318619 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:22.318619 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:22.318619 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:22.318619 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:22.319447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:22.319447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:22.319447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:22.319447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:22.319447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:22.319447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:22.319447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:22.319447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:22.319447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:22.319447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:22.319447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 08:24:22.319447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:22.319447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 08:24:22.319447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:22.319447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 420/847] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 08:24:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:22.320240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57ed2f-4ee5-41f8-b39b-3ff1e00dd89c None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:22.320240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57ed2f-4ee5-41f8-b39b-3ff1e00dd89c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:22.320361 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57ed2f-4ee5-41f8-b39b-3ff1e00dd89c None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:22.321584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 421/848] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:22.322260 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57ed2f-4ee5-41f8-b39b-3ff1e00dd89c None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:22.356905 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:22.388582 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23a271-35e5-4422-84e9-fc85fad7eca3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:22.423234 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57ed2f-4ee5-41f8-b39b-3ff1e00dd89c None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:22.425587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:22.428566 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad2a62-eefe-49ed-9302-5b67177584db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:22.429223 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad2a62-eefe-49ed-9302-5b67177584db None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:22.429500 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad2a62-eefe-49ed-9302-5b67177584db None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:22.429764 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad2a62-eefe-49ed-9302-5b67177584db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:22.440842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad2a62-eefe-49ed-9302-5b67177584db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:22.442303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad2a62-eefe-49ed-9302-5b67177584db None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:22.456033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-f7ad2a62-eefe-49ed-9302-5b67177584db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:22.456033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:22.456033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:22.456033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:22.456033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:22.456033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:22.456033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:22.456033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:22.456033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:22.456033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:22.456033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:22.456033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:22.456033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:22.456033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:22.456033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:22.456033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:22.457117 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:22.457117 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:22.457117 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:22.457117 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:22.457117 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:22.457117 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:22.457117 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:22.457117 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:22.457117 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:22.457117 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:22.457117 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:22.457117 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:22.457117 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:22.457117 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:22.457117 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:22.457117 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:22.457117 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:22.458597 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:22.458597 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:22.458597 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:22.458597 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 428/849] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:22.458597 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad2a62-eefe-49ed-9302-5b67177584db None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:22.462487 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547ec6e-7892-470c-b025-45a7fc1a373d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:22.463103 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547ec6e-7892-470c-b025-45a7fc1a373d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:22.463386 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547ec6e-7892-470c-b025-45a7fc1a373d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:22.463656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547ec6e-7892-470c-b025-45a7fc1a373d None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:22.464286 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7547ec6e-7892-470c-b025-45a7fc1a373d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:22.464585 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7547ec6e-7892-470c-b025-45a7fc1a373d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:22.465958 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7547ec6e-7892-470c-b025-45a7fc1a373d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:22.470581 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 422/850] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Aug 8 08:24:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:22.471709 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a6f89-65b6-451f-8da8-78912a196006 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:22.471994 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a6f89-65b6-451f-8da8-78912a196006 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:22.472266 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a6f89-65b6-451f-8da8-78912a196006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:22.489724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-ca8a6f89-65b6-451f-8da8-78912a196006 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:22.490890 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ca8a6f89-65b6-451f-8da8-78912a196006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:22.556181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad2a62-eefe-49ed-9302-5b67177584db None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:22.557677 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 423/851] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:22 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:22.558509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a6f89-65b6-451f-8da8-78912a196006 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:22.576750 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:22.592513 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c7b5a9-6bbf-46b5-b00a-571c3b9a3b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:22.593123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c7b5a9-6bbf-46b5-b00a-571c3b9a3b63 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:22.593428 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c7b5a9-6bbf-46b5-b00a-571c3b9a3b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:22.593933 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c7b5a9-6bbf-46b5-b00a-571c3b9a3b63 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:22.594314 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c7b5a9-6bbf-46b5-b00a-571c3b9a3b63 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:22.594610 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c7b5a9-6bbf-46b5-b00a-571c3b9a3b63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:22.595996 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c7b5a9-6bbf-46b5-b00a-571c3b9a3b63 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:22.600587 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 429/852] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 08:24:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:22.601395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c7b5a9-6bbf-46b5-b00a-571c3b9a3b63 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:22.654624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:22.658402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a6f89-65b6-451f-8da8-78912a196006 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:22.691780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7faa8e6-3c1b-4a91-988d-e724ec2c3dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:22.692382 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7faa8e6-3c1b-4a91-988d-e724ec2c3dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:22.692662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7faa8e6-3c1b-4a91-988d-e724ec2c3dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:22.692937 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7faa8e6-3c1b-4a91-988d-e724ec2c3dc7 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:22.693609 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7faa8e6-3c1b-4a91-988d-e724ec2c3dc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:22.693909 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7faa8e6-3c1b-4a91-988d-e724ec2c3dc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:22.695267 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7faa8e6-3c1b-4a91-988d-e724ec2c3dc7 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:22.700674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c7b5a9-6bbf-46b5-b00a-571c3b9a3b63 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:22.706361 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 424/853] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:22.707149 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7faa8e6-3c1b-4a91-988d-e724ec2c3dc7 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:22.736533 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e0d92-7ab1-4f97-8c21-a8a9e66ac912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:22.737135 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e0d92-7ab1-4f97-8c21-a8a9e66ac912 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:22.737455 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e0d92-7ab1-4f97-8c21-a8a9e66ac912 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:22.737725 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e0d92-7ab1-4f97-8c21-a8a9e66ac912 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:22.738345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e0d92-7ab1-4f97-8c21-a8a9e66ac912 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:22.738643 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e0d92-7ab1-4f97-8c21-a8a9e66ac912 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:22.739993 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e0d92-7ab1-4f97-8c21-a8a9e66ac912 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:22.744286 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 430/854] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Aug 8 08:24:22 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:22.745051 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e0d92-7ab1-4f97-8c21-a8a9e66ac912 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:22.800678 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:22.821300 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7faa8e6-3c1b-4a91-988d-e724ec2c3dc7 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:22.860114 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e0d92-7ab1-4f97-8c21-a8a9e66ac912 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:22.861649 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be68ac-9104-4100-b5e0-0a846528e2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:22.862381 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be68ac-9104-4100-b5e0-0a846528e2c5 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:22.862664 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be68ac-9104-4100-b5e0-0a846528e2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:22.862933 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be68ac-9104-4100-b5e0-0a846528e2c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:22.874219 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be68ac-9104-4100-b5e0-0a846528e2c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:24:22.879590 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-d0be68ac-9104-4100-b5e0-0a846528e2c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:24:22.879590 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:22.879590 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:22.879590 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:22.879590 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:22.879590 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:22.879590 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:22.879590 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:22.879590 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:22.879590 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:22.879590 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:22.879590 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:22.879590 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:24:22.879590 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:24:22.879590 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:24:22.879590 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:24:22.880785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:24:22.880785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:24:22.880785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:24:22.880785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:24:22.880785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:22.880785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:22.880785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:22.880785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:22.880785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:22.880785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:22.880785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:22.880785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:22.880785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:22.880785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:22.880785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:22.880785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:22.881689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:22.881689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:22.881689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:22.881689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:22.881689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:24:22.881689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:24:22.881689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:24:22.881689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:24:22.881689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:24:22.881689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:24:22.881689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:24:22.881689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:22.881689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 425/855] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 08:24:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:22.882386 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be68ac-9104-4100-b5e0-0a846528e2c5 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:22.906428 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b2aec4-c3df-4a76-a66a-43c4e57c9541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:22.907454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2aec4-c3df-4a76-a66a-43c4e57c9541 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:22.907734 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2aec4-c3df-4a76-a66a-43c4e57c9541 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:22.907995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2aec4-c3df-4a76-a66a-43c4e57c9541 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:22.908447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:22.908986 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b2aec4-c3df-4a76-a66a-43c4e57c9541 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:22.909505 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b2aec4-c3df-4a76-a66a-43c4e57c9541 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:22.910693 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b2aec4-c3df-4a76-a66a-43c4e57c9541 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:22.915886 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 431/856] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 08:24:22 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:22.916673 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b2aec4-c3df-4a76-a66a-43c4e57c9541 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:23.025174 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be68ac-9104-4100-b5e0-0a846528e2c5 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:23.045957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b2aec4-c3df-4a76-a66a-43c4e57c9541 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:23.062192 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:23.087255 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d858a13-9be4-4fd8-ac8a-15ebe165dd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:23.088414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d858a13-9be4-4fd8-ac8a-15ebe165dd0c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:23.088867 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d858a13-9be4-4fd8-ac8a-15ebe165dd0c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:23.090092 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d858a13-9be4-4fd8-ac8a-15ebe165dd0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:23.091073 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44888250-32ac-4cdc-ae1c-8656f84510de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:23.091540 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44888250-32ac-4cdc-ae1c-8656f84510de None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:23.091914 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44888250-32ac-4cdc-ae1c-8656f84510de None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:23.093654 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44888250-32ac-4cdc-ae1c-8656f84510de None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:23.093654 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44888250-32ac-4cdc-ae1c-8656f84510de None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:23.093941 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44888250-32ac-4cdc-ae1c-8656f84510de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:23.095336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44888250-32ac-4cdc-ae1c-8656f84510de None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:23.098168 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d858a13-9be4-4fd8-ac8a-15ebe165dd0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:23.099974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d858a13-9be4-4fd8-ac8a-15ebe165dd0c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:23.105208 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 432/857] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 08:24:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:23.111457 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44888250-32ac-4cdc-ae1c-8656f84510de None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:23.113053 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-6d858a13-9be4-4fd8-ac8a-15ebe165dd0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:23.113053 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:23.113053 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:23.113053 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:23.113053 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:23.113053 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:23.113053 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:23.113053 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:23.113053 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:23.113053 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:23.113053 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:23.113053 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:23.113053 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:24:23.113053 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:24:23.113053 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:24:23.113974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:24:23.113974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:23.113974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:23.113974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:23.113974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:23.113974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:23.113974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:23.113974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:23.113974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:23.113974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:23.113974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:23.113974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:23.113974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:23.113974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:23.113974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:23.113974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:23.113974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:23.115258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:24:23.115258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:24:23.115258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:24:23.115258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:24:23.115258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:24:23.115258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:24:23.115258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:23.115258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:23.115258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 426/858] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:24:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:23.120225 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d858a13-9be4-4fd8-ac8a-15ebe165dd0c None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:23.210162 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:23.233341 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d858a13-9be4-4fd8-ac8a-15ebe165dd0c None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:23.256260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44888250-32ac-4cdc-ae1c-8656f84510de None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:23.282227 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8098e8-a948-408e-9168-6a4e18a3fb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:23.283664 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8098e8-a948-408e-9168-6a4e18a3fb08 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:23.284154 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8098e8-a948-408e-9168-6a4e18a3fb08 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:23.284889 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8098e8-a948-408e-9168-6a4e18a3fb08 None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:23.285831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8098e8-a948-408e-9168-6a4e18a3fb08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:23.286636 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8098e8-a948-408e-9168-6a4e18a3fb08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:23.288920 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8098e8-a948-408e-9168-6a4e18a3fb08 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:23.294801 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0fb157-9170-4030-88c3-62afca036bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:23.295641 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fb157-9170-4030-88c3-62afca036bfa None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:23.296051 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fb157-9170-4030-88c3-62afca036bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:23.296645 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fb157-9170-4030-88c3-62afca036bfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:23.309437 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 427/859] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:24:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:23.312334 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8098e8-a948-408e-9168-6a4e18a3fb08 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:23.315166 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0fb157-9170-4030-88c3-62afca036bfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:23.316627 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0fb157-9170-4030-88c3-62afca036bfa None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:23.334165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-bf0fb157-9170-4030-88c3-62afca036bfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:23.334165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:23.334165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:23.334165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:23.334165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:23.334165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:23.334165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:23.334165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:23.334165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:23.334165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:23.334165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:23.334165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:23.334165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:23.334165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:23.334165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:23.334165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:23.335168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:23.335168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:23.335168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:23.335168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:23.335168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:23.335168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:23.335168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:23.335168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:23.335168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:23.335168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:23.335168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:23.335168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:23.335168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:23.335168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:23.335168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:23.335168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:23.335168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:23.336059 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:23.336059 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:23.336059 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:23.339948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 433/860] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:24:23.341208 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0fb157-9170-4030-88c3-62afca036bfa None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:23.372011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:23.450730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8098e8-a948-408e-9168-6a4e18a3fb08 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:23.468022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0fb157-9170-4030-88c3-62afca036bfa None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:23.489386 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f6dbc-caba-4ff6-be24-56e0cf9a6399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:23.490157 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f6dbc-caba-4ff6-be24-56e0cf9a6399 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:23.490634 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f6dbc-caba-4ff6-be24-56e0cf9a6399 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:23.490964 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f6dbc-caba-4ff6-be24-56e0cf9a6399 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:23.491937 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f6dbc-caba-4ff6-be24-56e0cf9a6399 None admin] RBAC: Adding query filter params (role.id=c1e514a4b5424f959b069f36cbfd8147, scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4, user.id=e69c0e02ad944f2a85fc5318018e5cb0) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:23.492258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f6dbc-caba-4ff6-be24-56e0cf9a6399 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:23.493807 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f6dbc-caba-4ff6-be24-56e0cf9a6399 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:23.509199 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 428/861] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:24:23 2022] GET /identity/v3/role_assignments?user.id=e69c0e02ad944f2a85fc5318018e5cb0&scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4&role.id=c1e514a4b5424f959b069f36cbfd8147 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:23.522586 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40160941-f230-4692-8c92-8c4b61c45bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:23.523274 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40160941-f230-4692-8c92-8c4b61c45bff None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:23.523559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40160941-f230-4692-8c92-8c4b61c45bff None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:23.523874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40160941-f230-4692-8c92-8c4b61c45bff None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:23.524544 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40160941-f230-4692-8c92-8c4b61c45bff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:23.524846 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40160941-f230-4692-8c92-8c4b61c45bff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:23.526368 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40160941-f230-4692-8c92-8c4b61c45bff None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:23.548410 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 434/862] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:23.550031 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f6dbc-caba-4ff6-be24-56e0cf9a6399 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:23.553378 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:23.670284 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f6dbc-caba-4ff6-be24-56e0cf9a6399 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:23.707492 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c560d-9d78-4900-9352-340d92fabcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:23.708729 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c560d-9d78-4900-9352-340d92fabcab None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:23.709267 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c560d-9d78-4900-9352-340d92fabcab None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:23.709842 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c560d-9d78-4900-9352-340d92fabcab None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:23.717869 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c560d-9d78-4900-9352-340d92fabcab None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:23.719532 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c560d-9d78-4900-9352-340d92fabcab None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:23.731040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-d17c560d-9d78-4900-9352-340d92fabcab None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 08:24:23.731040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:23.731040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:23.731040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:23.731040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:23.731040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:23.731040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:23.731040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:23.731040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:23.731040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:23.731040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:23.731040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:23.731040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:24:23.731040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:24:23.731040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:24:23.731040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:24:23.732203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:23.732203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:23.732203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:24:23.732203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:23.732203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:24:23.732203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:23.732203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:23.732203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:23.732203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:23.732203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:23.732203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:23.732203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:23.732203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:23.732203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:23.732203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:23.732203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:23.732203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:23.732203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:23.733854 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:23.733854 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:23.733854 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:24:23.733854 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:24:23.733854 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:24:23.733854 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:24:23.733854 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:24:23.733854 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:24:23.733854 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 08:24:23.733854 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:23.735864 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 429/863] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Aug 8 08:24:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:23.741658 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40160941-f230-4692-8c92-8c4b61c45bff None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:23.831913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:23.847264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40160941-f230-4692-8c92-8c4b61c45bff None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:23.882853 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b89232-2e0d-491a-8d52-4a14dd050c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:23.883463 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b89232-2e0d-491a-8d52-4a14dd050c01 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:23.883742 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b89232-2e0d-491a-8d52-4a14dd050c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:23.884012 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b89232-2e0d-491a-8d52-4a14dd050c01 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:23.884826 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b89232-2e0d-491a-8d52-4a14dd050c01 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:23.885133 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b89232-2e0d-491a-8d52-4a14dd050c01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:23.886589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b89232-2e0d-491a-8d52-4a14dd050c01 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:23.907601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 435/864] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Aug 8 08:24:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:23.913330 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c560d-9d78-4900-9352-340d92fabcab None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:23.983942 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:24.032313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c560d-9d78-4900-9352-340d92fabcab None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:24.069588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b22bf1-fe33-4b2e-8359-019a1b2d9c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:24.070766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b22bf1-fe33-4b2e-8359-019a1b2d9c67 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:24.071245 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b22bf1-fe33-4b2e-8359-019a1b2d9c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:24.071711 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b22bf1-fe33-4b2e-8359-019a1b2d9c67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:24.146964 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b22bf1-fe33-4b2e-8359-019a1b2d9c67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:24.148778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b22bf1-fe33-4b2e-8359-019a1b2d9c67 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:24.211388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:24.257332 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 430/865] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:24:23 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 08:24:24.259959 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b89232-2e0d-491a-8d52-4a14dd050c01 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:24.365294 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b89232-2e0d-491a-8d52-4a14dd050c01 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:24.401088 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd63cf-df1c-4ffd-8bb5-76e464e3ea58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:24.401753 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd63cf-df1c-4ffd-8bb5-76e464e3ea58 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:24.402030 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd63cf-df1c-4ffd-8bb5-76e464e3ea58 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:24.402300 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd63cf-df1c-4ffd-8bb5-76e464e3ea58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:24.413461 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd63cf-df1c-4ffd-8bb5-76e464e3ea58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:24.414980 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd63cf-df1c-4ffd-8bb5-76e464e3ea58 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:24.425930 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:24.432315 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-01dd63cf-df1c-4ffd-8bb5-76e464e3ea58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:24.432315 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:24.432315 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:24.432315 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:24.432315 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:24.432315 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:24.432315 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:24.432315 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:24.432315 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:24.432315 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:24.432315 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:24.432315 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:24.432315 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:24.432315 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:24.432315 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:24.432315 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:24.433444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:24.433444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:24.433444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:24.433444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:24.433444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:24.433444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:24.433444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:24.433444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:24.433444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:24.433444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:24.433444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:24.433444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:24.433444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:24.433444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:24.433444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:24.433444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:24.433444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:24.434392 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:24.434392 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:24.434392 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:24.434392 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 436/866] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:24.439547 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b22bf1-fe33-4b2e-8359-019a1b2d9c67 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:24.544766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b22bf1-fe33-4b2e-8359-019a1b2d9c67 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:24.580903 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b67b26-385d-4049-9895-ac2e9a726cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:24.581550 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b67b26-385d-4049-9895-ac2e9a726cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:24.581828 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b67b26-385d-4049-9895-ac2e9a726cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:24.582093 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b67b26-385d-4049-9895-ac2e9a726cd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:24.582729 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b67b26-385d-4049-9895-ac2e9a726cd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:24.583275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b67b26-385d-4049-9895-ac2e9a726cd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:24.584620 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b67b26-385d-4049-9895-ac2e9a726cd9 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:24.600803 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 431/867] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:24.605339 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd63cf-df1c-4ffd-8bb5-76e464e3ea58 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:24.710347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd63cf-df1c-4ffd-8bb5-76e464e3ea58 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:24.745508 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935a9b1-459c-4e04-98aa-8e9edf1be432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:24.746144 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935a9b1-459c-4e04-98aa-8e9edf1be432 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:24.746423 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935a9b1-459c-4e04-98aa-8e9edf1be432 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:24.746693 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935a9b1-459c-4e04-98aa-8e9edf1be432 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:24.754263 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c935a9b1-459c-4e04-98aa-8e9edf1be432 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:24.755717 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c935a9b1-459c-4e04-98aa-8e9edf1be432 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:24.767865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-c935a9b1-459c-4e04-98aa-8e9edf1be432 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:24.767865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:24.767865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:24.767865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:24.767865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:24.767865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:24.767865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:24.767865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:24.767865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:24.767865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:24.767865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:24.767865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:24.767865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:24:24.767865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:24:24.767865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:24:24.770002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:24:24.770002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:24.770002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:24.770002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:24.770002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:24.770002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:24.770002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:24.770002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:24.770002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:24.770002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:24.770002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:24.770002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:24.770002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:24.770002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:24.770002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:24.770002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:24.770002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:24.772066 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:24:24.772066 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:24:24.772066 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:24:24.772066 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:24:24.772066 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:24:24.772066 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:24:24.772066 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:24.772066 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:24.772066 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 437/868] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:24:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:24.775578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b67b26-385d-4049-9895-ac2e9a726cd9 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:24.803540 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:24.810880 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb356d-1ba7-4190-b9ae-deffd846051a None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:24.811673 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb356d-1ba7-4190-b9ae-deffd846051a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:24.811959 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb356d-1ba7-4190-b9ae-deffd846051a None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:24.813518 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 438/869] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:24.818335 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b8547-42c8-4c61-9e58-a4f6b49d5c7b None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:24.827929 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b8547-42c8-4c61-9e58-a4f6b49d5c7b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:24.827929 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b8547-42c8-4c61-9e58-a4f6b49d5c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:24.838209 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-128b8547-42c8-4c61-9e58-a4f6b49d5c7b None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:24.840003 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-128b8547-42c8-4c61-9e58-a4f6b49d5c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:24.893128 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b67b26-385d-4049-9895-ac2e9a726cd9 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:24.894191 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:24.911646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 439/870] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 08:24:24.931408 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae15373-e02e-4b00-b869-a7e6a516cdc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:24.932071 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae15373-e02e-4b00-b869-a7e6a516cdc4 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:24.932350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae15373-e02e-4b00-b869-a7e6a516cdc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:24.932628 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae15373-e02e-4b00-b869-a7e6a516cdc4 None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:24.933312 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b8547-42c8-4c61-9e58-a4f6b49d5c7b None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:24.933423 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae15373-e02e-4b00-b869-a7e6a516cdc4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:24.933641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae15373-e02e-4b00-b869-a7e6a516cdc4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:24.935153 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae15373-e02e-4b00-b869-a7e6a516cdc4 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:24.954805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 432/871] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:24:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:24.956242 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae15373-e02e-4b00-b869-a7e6a516cdc4 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:25.037903 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b8547-42c8-4c61-9e58-a4f6b49d5c7b None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:25.044684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:25.064013 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae15373-e02e-4b00-b869-a7e6a516cdc4 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:25.071983 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765ff9d-cf9c-4212-a83f-1e2cfeb15755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:25.072591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765ff9d-cf9c-4212-a83f-1e2cfeb15755 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:25.072877 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765ff9d-cf9c-4212-a83f-1e2cfeb15755 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:25.073165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765ff9d-cf9c-4212-a83f-1e2cfeb15755 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:25.073800 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765ff9d-cf9c-4212-a83f-1e2cfeb15755 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:25.075170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765ff9d-cf9c-4212-a83f-1e2cfeb15755 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:25.098971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa799fb-e7a7-4490-aa46-91aac4a18ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:25.099571 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa799fb-e7a7-4490-aa46-91aac4a18ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:25.099850 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa799fb-e7a7-4490-aa46-91aac4a18ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:25.100114 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa799fb-e7a7-4490-aa46-91aac4a18ec2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:25.100875 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa799fb-e7a7-4490-aa46-91aac4a18ec2 None admin] RBAC: Adding query filter params (role.id=402c3e6fe3364285bd72784a297db11e, scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4, user.id=7f3422c6f9a041ceba851b71bd15d0df) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:25.101210 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa799fb-e7a7-4490-aa46-91aac4a18ec2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:25.102607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa799fb-e7a7-4490-aa46-91aac4a18ec2 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:25.108927 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 433/872] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:24:24 2022] GET /identity/v3/role_assignments?user.id=7f3422c6f9a041ceba851b71bd15d0df&scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4&role.id=402c3e6fe3364285bd72784a297db11e => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:25.110245 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5e41a-23ac-406a-b030-e3d750a30837 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:25.110535 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5e41a-23ac-406a-b030-e3d750a30837 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:25.110802 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5e41a-23ac-406a-b030-e3d750a30837 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:25.112157 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 434/873] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:25.118146 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b6630-dec9-437c-84f1-0420fffee804 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:25.118461 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b6630-dec9-437c-84f1-0420fffee804 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:25.118716 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b6630-dec9-437c-84f1-0420fffee804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:25.137933 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-616b6630-dec9-437c-84f1-0420fffee804 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:25.139122 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-616b6630-dec9-437c-84f1-0420fffee804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:25.141854 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 440/874] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Aug 8 08:24:24 2022] POST /identity/v3/services => generated 269 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:24:25.239286 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 435/875] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:25 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:25.258087 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765ff9d-cf9c-4212-a83f-1e2cfeb15755 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:25.315164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:25.374544 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765ff9d-cf9c-4212-a83f-1e2cfeb15755 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:25.419805 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fedadf-f5c3-44c4-ba44-1050044d094a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:25.420443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fedadf-f5c3-44c4-ba44-1050044d094a None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:25.421467 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fedadf-f5c3-44c4-ba44-1050044d094a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:25.421851 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fedadf-f5c3-44c4-ba44-1050044d094a None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:25.422571 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fedadf-f5c3-44c4-ba44-1050044d094a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:25.423964 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fedadf-f5c3-44c4-ba44-1050044d094a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:25.520901 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 441/876] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Aug 8 08:24:25 2022] POST /identity/v3/services => generated 251 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:24:25.747081 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47273fd-9c4a-41e5-aea2-685210cd52f5 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:25.747451 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47273fd-9c4a-41e5-aea2-685210cd52f5 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:25.747650 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47273fd-9c4a-41e5-aea2-685210cd52f5 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:25.749373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 436/877] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:25.754642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a97eea-b483-4052-8cce-04e370ede4fd None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:25.754927 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a97eea-b483-4052-8cce-04e370ede4fd None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:25.755200 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a97eea-b483-4052-8cce-04e370ede4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:25.774484 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d7a97eea-b483-4052-8cce-04e370ede4fd None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:25.775671 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d7a97eea-b483-4052-8cce-04e370ede4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:25.851559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 442/878] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:25 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:25.874349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47273fd-9c4a-41e5-aea2-685210cd52f5 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:25.950921 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:25.982162 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47273fd-9c4a-41e5-aea2-685210cd52f5 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:26.019010 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e95594-e8bb-4f9c-95df-8de8cc710c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:26.019559 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e95594-e8bb-4f9c-95df-8de8cc710c27 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:26.019834 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e95594-e8bb-4f9c-95df-8de8cc710c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:26.020104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e95594-e8bb-4f9c-95df-8de8cc710c27 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:26.020679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e95594-e8bb-4f9c-95df-8de8cc710c27 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:26.022050 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e95594-e8bb-4f9c-95df-8de8cc710c27 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:26.031401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-e2e95594-e8bb-4f9c-95df-8de8cc710c27 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 08:24:26.031401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:26.031401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 08:24:26.031401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 08:24:26.031401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 08:24:26.031401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 08:24:26.031401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 08:24:26.031401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:26.031401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 08:24:26.031401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:26.031401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:24:26.031401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:26.031401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:26.031401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:26.031401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:26.031401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:26.031401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:26.032392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:26.032392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:26.032392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:26.032392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:26.032392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:26.032392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:26.032392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 08:24:26.032392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 08:24:26.032392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 08:24:26.032392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 08:24:26.032392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:26.032392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:26.032392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:26.032392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:26.032392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:26.032392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:26.033262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:26.033262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:26.033262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:26.033262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:26.033262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:26.033262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:26.033262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:26.033262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:26.033262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:26.033262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:26.033262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 08:24:26.033262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:26.033262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 08:24:26.033262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:26.033262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 437/879] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Aug 8 08:24:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:26.038788 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a97eea-b483-4052-8cce-04e370ede4fd None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:26.072981 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:26.146575 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a97eea-b483-4052-8cce-04e370ede4fd None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:26.186016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34bb07-78af-415f-8c9d-a69dc6570664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:26.186653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34bb07-78af-415f-8c9d-a69dc6570664 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:26.186927 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34bb07-78af-415f-8c9d-a69dc6570664 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:26.187196 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34bb07-78af-415f-8c9d-a69dc6570664 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:26.187817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34bb07-78af-415f-8c9d-a69dc6570664 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:26.188110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34bb07-78af-415f-8c9d-a69dc6570664 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:26.189491 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34bb07-78af-415f-8c9d-a69dc6570664 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:26.194646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 443/880] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Aug 8 08:24:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:26.200095 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e95594-e8bb-4f9c-95df-8de8cc710c27 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:26.302813 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e95594-e8bb-4f9c-95df-8de8cc710c27 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:26.340503 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8235f3-68f8-468f-836e-8a94b6d9e38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:26.341107 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8235f3-68f8-468f-836e-8a94b6d9e38d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:26.341411 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8235f3-68f8-468f-836e-8a94b6d9e38d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:26.341678 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8235f3-68f8-468f-836e-8a94b6d9e38d None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:26.342303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8235f3-68f8-468f-836e-8a94b6d9e38d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:26.342625 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8235f3-68f8-468f-836e-8a94b6d9e38d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:26.343966 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8235f3-68f8-468f-836e-8a94b6d9e38d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:26.348783 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:26.349083 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 438/881] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Aug 8 08:24:26 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:27.223205 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bcaea2-9a8e-44da-939d-029d43885c99 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:27.223765 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bcaea2-9a8e-44da-939d-029d43885c99 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:27.223765 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bcaea2-9a8e-44da-939d-029d43885c99 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:27.225317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 444/882] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:27.230584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef335ee-b22b-482b-b60d-4b73640f0c11 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:27.231425 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef335ee-b22b-482b-b60d-4b73640f0c11 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:27.231914 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef335ee-b22b-482b-b60d-4b73640f0c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:27.250811 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-fef335ee-b22b-482b-b60d-4b73640f0c11 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:27.252329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-fef335ee-b22b-482b-b60d-4b73640f0c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:27.291464 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29547f2e-211c-462d-b3d2-ec2f0352b41e None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:27.292097 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29547f2e-211c-462d-b3d2-ec2f0352b41e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:27.292582 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29547f2e-211c-462d-b3d2-ec2f0352b41e None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:27.294381 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 445/883] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 08:24:27.299584 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd0a28-480d-46e5-826a-46415488de16 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:27.300116 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd0a28-480d-46e5-826a-46415488de16 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:27.300578 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd0a28-480d-46e5-826a-46415488de16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:27.319647 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-62bd0a28-480d-46e5-826a-46415488de16 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:27.321262 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-62bd0a28-480d-46e5-826a-46415488de16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:27.323864 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 439/884] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:27 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:27.346841 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef335ee-b22b-482b-b60d-4b73640f0c11 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:27.404043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 446/885] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:27 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 08:24:27.437209 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd0a28-480d-46e5-826a-46415488de16 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:27.465545 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef335ee-b22b-482b-b60d-4b73640f0c11 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:27.475085 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:27.498758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:27.508482 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26063c86-83fa-4383-906b-40c4c86146a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:27.513444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26063c86-83fa-4383-906b-40c4c86146a1 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:27.513444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26063c86-83fa-4383-906b-40c4c86146a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:27.513444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26063c86-83fa-4383-906b-40c4c86146a1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:27.513444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26063c86-83fa-4383-906b-40c4c86146a1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:27.513444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26063c86-83fa-4383-906b-40c4c86146a1 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:27.523971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-26063c86-83fa-4383-906b-40c4c86146a1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 08:24:27.523971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:27.523971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 08:24:27.523971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 08:24:27.523971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 08:24:27.523971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 08:24:27.523971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 08:24:27.523971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:27.523971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 08:24:27.523971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:27.523971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:24:27.523971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:27.523971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:27.523971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:27.523971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:27.523971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:27.524941 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:27.524941 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:27.524941 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:27.524941 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:27.524941 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:27.524941 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:27.524941 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:27.524941 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 08:24:27.524941 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 08:24:27.524941 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 08:24:27.524941 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 08:24:27.524941 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:27.524941 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:27.524941 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:27.524941 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:27.524941 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:27.524941 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:27.525862 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:27.525862 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:27.525862 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:27.525862 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:27.525862 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:27.525862 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:27.525862 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:27.525862 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:27.525862 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:27.525862 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:27.525862 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 08:24:27.525862 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:27.525862 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 08:24:27.525862 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:27.529264 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 440/886] 10.222.0.33 () {56 vars in 1132 bytes} [Mon Aug 8 08:24:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:27.531815 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26063c86-83fa-4383-906b-40c4c86146a1 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:27.569859 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd0a28-480d-46e5-826a-46415488de16 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:27.614633 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7aad5-7a46-4d84-835a-6cd2147dac2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:27.615901 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7aad5-7a46-4d84-835a-6cd2147dac2a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:27.616385 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7aad5-7a46-4d84-835a-6cd2147dac2a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:27.617184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7aad5-7a46-4d84-835a-6cd2147dac2a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:27.631172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7aad5-7a46-4d84-835a-6cd2147dac2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:24:27.637572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-47b7aad5-7a46-4d84-835a-6cd2147dac2a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 08:24:27.637572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:27.637572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:27.637572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:27.637572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:27.637572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:27.637572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:27.637572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:27.637572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:27.637572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:27.637572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:27.637572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:27.637572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:24:27.637572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:24:27.637572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:24:27.637572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:24:27.638507 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:24:27.638507 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:24:27.638507 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:24:27.638507 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:24:27.638507 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:27.638507 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:27.638507 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:27.638507 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:27.638507 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:27.638507 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:27.638507 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:27.638507 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:27.638507 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:27.638507 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:27.638507 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:27.638507 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:27.639874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:27.639874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:27.639874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:27.639874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:27.639874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:24:27.639874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:24:27.639874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:24:27.639874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:24:27.639874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:24:27.639874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:24:27.639874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 08:24:27.639874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:27.642550 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 447/887] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Aug 8 08:24:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:24:27.648283 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7aad5-7a46-4d84-835a-6cd2147dac2a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:27.658235 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26063c86-83fa-4383-906b-40c4c86146a1 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:27.694352 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c81bb-bff5-47b3-a6e1-a20d327ad41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:27.694958 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c81bb-bff5-47b3-a6e1-a20d327ad41a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:27.695234 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c81bb-bff5-47b3-a6e1-a20d327ad41a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:27.695501 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c81bb-bff5-47b3-a6e1-a20d327ad41a None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:27.696141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c81bb-bff5-47b3-a6e1-a20d327ad41a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:27.696433 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c81bb-bff5-47b3-a6e1-a20d327ad41a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:27.699646 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c81bb-bff5-47b3-a6e1-a20d327ad41a None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:27.710633 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 441/888] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Aug 8 08:24:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:27.712761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c81bb-bff5-47b3-a6e1-a20d327ad41a None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:27.761657 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:27.776279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7aad5-7a46-4d84-835a-6cd2147dac2a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:27.785252 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:27.814348 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b986540-cff0-4d2c-be3d-bc9a153291bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:27.814961 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b986540-cff0-4d2c-be3d-bc9a153291bc None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:27.815239 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b986540-cff0-4d2c-be3d-bc9a153291bc None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:27.815515 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b986540-cff0-4d2c-be3d-bc9a153291bc None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:27.816308 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b986540-cff0-4d2c-be3d-bc9a153291bc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:27.816618 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b986540-cff0-4d2c-be3d-bc9a153291bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:27.818033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b986540-cff0-4d2c-be3d-bc9a153291bc None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:27.826694 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 448/889] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Aug 8 08:24:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:27.827920 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f4dcc-23dd-4fe0-9c34-bdd778da7024 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:27.828202 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f4dcc-23dd-4fe0-9c34-bdd778da7024 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:27.828473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f4dcc-23dd-4fe0-9c34-bdd778da7024 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:27.829847 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 449/890] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:27.834591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f4dcc-23dd-4fe0-9c34-bdd778da7024 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:27.835083 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c81bb-bff5-47b3-a6e1-a20d327ad41a None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:27.872136 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ced08-107c-4c56-81dd-48a8415a4a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:27.872745 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ced08-107c-4c56-81dd-48a8415a4a5e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:27.873026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ced08-107c-4c56-81dd-48a8415a4a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:27.873331 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ced08-107c-4c56-81dd-48a8415a4a5e None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:27.874214 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ced08-107c-4c56-81dd-48a8415a4a5e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:27.874524 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ced08-107c-4c56-81dd-48a8415a4a5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:27.875945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ced08-107c-4c56-81dd-48a8415a4a5e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:27.881173 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 442/891] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Aug 8 08:24:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:27.882414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f7edb-736a-4987-a224-45f06068edd7 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:27.882708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f7edb-736a-4987-a224-45f06068edd7 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:27.882987 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f7edb-736a-4987-a224-45f06068edd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:27.901445 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-b70f7edb-736a-4987-a224-45f06068edd7 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:27.902732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b70f7edb-736a-4987-a224-45f06068edd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:27.950689 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f4dcc-23dd-4fe0-9c34-bdd778da7024 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:27.980291 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 443/892] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:27 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:27.981536 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f7edb-736a-4987-a224-45f06068edd7 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:28.002914 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15575fe4-b6c4-4beb-b3c6-be214b4eb336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:28.004481 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15575fe4-b6c4-4beb-b3c6-be214b4eb336 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:28.005114 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15575fe4-b6c4-4beb-b3c6-be214b4eb336 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:28.005702 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15575fe4-b6c4-4beb-b3c6-be214b4eb336 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:28.022113 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15575fe4-b6c4-4beb-b3c6-be214b4eb336 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:28.024320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15575fe4-b6c4-4beb-b3c6-be214b4eb336 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:28.036052 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:28.036673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:28.042939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-15575fe4-b6c4-4beb-b3c6-be214b4eb336 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:28.042939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:28.042939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:28.042939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:28.042939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:28.042939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:28.042939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:28.042939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:28.042939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:28.042939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:28.042939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:28.042939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:28.042939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:28.042939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:28.042939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:28.042939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:28.050391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:28.050391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:28.050391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:28.050391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:28.050391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:28.050391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:28.050391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:28.050391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:28.050391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:28.050391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:28.050391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:28.050391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:28.050391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:28.050391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:28.050391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:28.050391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:28.050391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:28.051467 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:28.051467 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:28.051467 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:28.051467 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 450/893] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:28.051467 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15575fe4-b6c4-4beb-b3c6-be214b4eb336 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:28.088724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f7edb-736a-4987-a224-45f06068edd7 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:28.125467 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd8a94-8cc3-4d8a-89f7-0fbf76af8231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:28.126232 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd8a94-8cc3-4d8a-89f7-0fbf76af8231 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:28.126607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd8a94-8cc3-4d8a-89f7-0fbf76af8231 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:28.126974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd8a94-8cc3-4d8a-89f7-0fbf76af8231 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:28.127767 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd8a94-8cc3-4d8a-89f7-0fbf76af8231 None admin] RBAC: Adding query filter params (interface=public, service_id=b7fc7882a0b54b4884577dacea5b7641, region_id=RegionOne) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:28.128172 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd8a94-8cc3-4d8a-89f7-0fbf76af8231 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:28.129989 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd8a94-8cc3-4d8a-89f7-0fbf76af8231 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:28.139803 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 444/894] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Aug 8 08:24:27 2022] GET /identity/v3/endpoints?service_id=b7fc7882a0b54b4884577dacea5b7641&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:28.139803 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd8a94-8cc3-4d8a-89f7-0fbf76af8231 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:28.156708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15575fe4-b6c4-4beb-b3c6-be214b4eb336 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:28.192156 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f20867-483d-419f-aa94-e60c20b4fead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:28.192787 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f20867-483d-419f-aa94-e60c20b4fead None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:28.193064 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f20867-483d-419f-aa94-e60c20b4fead None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:28.193365 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f20867-483d-419f-aa94-e60c20b4fead None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:28.193947 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f20867-483d-419f-aa94-e60c20b4fead None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:28.195308 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f20867-483d-419f-aa94-e60c20b4fead None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:28.205342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-c7f20867-483d-419f-aa94-e60c20b4fead None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 08:24:28.205342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:28.205342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 08:24:28.205342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 08:24:28.205342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 08:24:28.205342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 08:24:28.205342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 08:24:28.205342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:28.205342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 08:24:28.205342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:28.205342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:24:28.205342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:28.205342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:28.205342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:28.205342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:28.205342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:28.205342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:28.206325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:28.206325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:28.206325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:28.206325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:28.206325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:28.206325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:28.206325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 08:24:28.206325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 08:24:28.206325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 08:24:28.206325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 08:24:28.206325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:28.206325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:28.206325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:28.206325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:28.206325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:28.206325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:28.207169 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:28.207169 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:28.207169 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:28.207169 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:28.207169 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:28.207169 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:28.207169 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:28.207169 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:28.207169 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:28.207169 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:28.207169 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 08:24:28.207169 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:28.207169 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 08:24:28.207169 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:28.207169 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 451/895] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 08:24:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:28.208940 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f20867-483d-419f-aa94-e60c20b4fead None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:28.254733 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd8a94-8cc3-4d8a-89f7-0fbf76af8231 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:28.272540 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:28.275104 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:28.293773 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30ed84-fcc7-4eb5-8a25-36b9797b2d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:28.294780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30ed84-fcc7-4eb5-8a25-36b9797b2d1b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:28.296114 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30ed84-fcc7-4eb5-8a25-36b9797b2d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:28.296927 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30ed84-fcc7-4eb5-8a25-36b9797b2d1b None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:28.297856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30ed84-fcc7-4eb5-8a25-36b9797b2d1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:28.298661 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30ed84-fcc7-4eb5-8a25-36b9797b2d1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:28.300481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30ed84-fcc7-4eb5-8a25-36b9797b2d1b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:28.316612 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f20867-483d-419f-aa94-e60c20b4fead None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:28.317840 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 445/896] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:28.318939 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30ed84-fcc7-4eb5-8a25-36b9797b2d1b None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:28.354942 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3875b-289e-48ac-b1d8-6ba2b1240894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:28.355917 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3875b-289e-48ac-b1d8-6ba2b1240894 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:28.356340 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3875b-289e-48ac-b1d8-6ba2b1240894 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:28.356753 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3875b-289e-48ac-b1d8-6ba2b1240894 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:28.357524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb3875b-289e-48ac-b1d8-6ba2b1240894 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:28.359579 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb3875b-289e-48ac-b1d8-6ba2b1240894 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:28.377175 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 452/897] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Aug 8 08:24:28 2022] GET /identity/v3/services => generated 1268 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:24:28.377175 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3875b-289e-48ac-b1d8-6ba2b1240894 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:28.439317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30ed84-fcc7-4eb5-8a25-36b9797b2d1b None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:28.480331 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc3b441-1c0b-4c38-8e93-5e919f0c683d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:28.480952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc3b441-1c0b-4c38-8e93-5e919f0c683d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:28.481256 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc3b441-1c0b-4c38-8e93-5e919f0c683d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:28.481528 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc3b441-1c0b-4c38-8e93-5e919f0c683d None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:28.482285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc3b441-1c0b-4c38-8e93-5e919f0c683d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:28.482587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc3b441-1c0b-4c38-8e93-5e919f0c683d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:28.484137 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc3b441-1c0b-4c38-8e93-5e919f0c683d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:28.494509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 446/898] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 08:24:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:28.494509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de0be8-c859-42b4-b799-791b27a77a05 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:28.494509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de0be8-c859-42b4-b799-791b27a77a05 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:28.494509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de0be8-c859-42b4-b799-791b27a77a05 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:28.494509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 447/899] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:28.498310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de0be8-c859-42b4-b799-791b27a77a05 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:28.510421 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3875b-289e-48ac-b1d8-6ba2b1240894 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:28.544610 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:28.551092 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:28.558119 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b0fe9-109f-49ae-8227-950c801e0567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:28.558774 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b0fe9-109f-49ae-8227-950c801e0567 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:28.559058 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b0fe9-109f-49ae-8227-950c801e0567 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:28.559631 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b0fe9-109f-49ae-8227-950c801e0567 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:28.571390 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b0fe9-109f-49ae-8227-950c801e0567 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:28.572860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b0fe9-109f-49ae-8227-950c801e0567 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:28.585731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-936b0fe9-109f-49ae-8227-950c801e0567 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 08:24:28.585731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:28.585731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:28.585731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:28.585731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:28.585731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:28.585731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:28.585731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:28.585731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:28.585731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:28.585731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:28.585731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:28.585731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:24:28.585731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:24:28.585731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:24:28.585731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:24:28.586726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:28.586726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:28.586726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:24:28.586726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:28.586726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:24:28.586726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:28.586726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:28.586726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:28.586726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:28.586726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:28.586726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:28.586726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:28.586726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:28.586726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:28.586726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:28.586726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:28.586726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:28.586726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:28.588273 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:28.588273 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:28.588273 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:24:28.588273 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:24:28.588273 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:24:28.588273 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:24:28.588273 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:24:28.588273 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:24:28.588273 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 08:24:28.588273 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:28.588273 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 453/900] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Aug 8 08:24:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:28.588273 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c05ce-5ea8-443b-bad7-e7aca3d25038 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:28.588273 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c05ce-5ea8-443b-bad7-e7aca3d25038 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:28.588978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c05ce-5ea8-443b-bad7-e7aca3d25038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:28.606883 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-678c05ce-5ea8-443b-bad7-e7aca3d25038 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:28.608152 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-678c05ce-5ea8-443b-bad7-e7aca3d25038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:28.619726 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de0be8-c859-42b4-b799-791b27a77a05 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:28.660997 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf21296-2072-4352-8af5-6a9d000a9ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:28.661666 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf21296-2072-4352-8af5-6a9d000a9ffc None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:28.661944 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf21296-2072-4352-8af5-6a9d000a9ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:28.662214 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf21296-2072-4352-8af5-6a9d000a9ffc None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:28.662844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf21296-2072-4352-8af5-6a9d000a9ffc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:28.663145 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf21296-2072-4352-8af5-6a9d000a9ffc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:28.664757 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf21296-2072-4352-8af5-6a9d000a9ffc None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:28.677169 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 448/901] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 08:24:28 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:28.677169 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf21296-2072-4352-8af5-6a9d000a9ffc None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:28.687695 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 454/902] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:28 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:28.688488 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c05ce-5ea8-443b-bad7-e7aca3d25038 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:28.790911 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf21296-2072-4352-8af5-6a9d000a9ffc None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:28.793773 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:28.795245 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c05ce-5ea8-443b-bad7-e7aca3d25038 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:28.814787 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:28.827520 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe2e20-918d-42a4-9805-aea05da555b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:28.828142 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe2e20-918d-42a4-9805-aea05da555b4 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:28.828422 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe2e20-918d-42a4-9805-aea05da555b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:28.828697 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe2e20-918d-42a4-9805-aea05da555b4 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:28.830638 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe2e20-918d-42a4-9805-aea05da555b4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:28.830638 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe2e20-918d-42a4-9805-aea05da555b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:28.831184 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe2e20-918d-42a4-9805-aea05da555b4 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:28.836855 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6fb5de-71cf-4b20-bdeb-b1c85703ca24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:28.837528 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6fb5de-71cf-4b20-bdeb-b1c85703ca24 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:28.837832 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6fb5de-71cf-4b20-bdeb-b1c85703ca24 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:28.838110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6fb5de-71cf-4b20-bdeb-b1c85703ca24 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:28.838814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6fb5de-71cf-4b20-bdeb-b1c85703ca24 None admin] RBAC: Adding query filter params (interface=public, service_id=541c213708f34701abc700b1b7d8bc3a, region_id=RegionOne) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:28.839121 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6fb5de-71cf-4b20-bdeb-b1c85703ca24 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:28.840604 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6fb5de-71cf-4b20-bdeb-b1c85703ca24 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:28.848380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 455/903] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Aug 8 08:24:28 2022] GET /identity/v3/endpoints?service_id=541c213708f34701abc700b1b7d8bc3a&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:28.852830 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6fb5de-71cf-4b20-bdeb-b1c85703ca24 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:28.855294 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 449/904] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Aug 8 08:24:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:28.856121 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe2e20-918d-42a4-9805-aea05da555b4 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:28.961249 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6fb5de-71cf-4b20-bdeb-b1c85703ca24 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:28.961477 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe2e20-918d-42a4-9805-aea05da555b4 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:28.995478 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d591f93c-6125-49b3-8c23-891c6e382807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:28.996081 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d591f93c-6125-49b3-8c23-891c6e382807 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:28.996357 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d591f93c-6125-49b3-8c23-891c6e382807 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:28.996620 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d591f93c-6125-49b3-8c23-891c6e382807 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:28.997248 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d591f93c-6125-49b3-8c23-891c6e382807 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:28.998350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef865fb-9f66-470d-9de7-a17de8e1f591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:28.998691 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d591f93c-6125-49b3-8c23-891c6e382807 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:28.999250 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef865fb-9f66-470d-9de7-a17de8e1f591 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:28.999534 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef865fb-9f66-470d-9de7-a17de8e1f591 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:28.999926 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef865fb-9f66-470d-9de7-a17de8e1f591 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:29.000689 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef865fb-9f66-470d-9de7-a17de8e1f591 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:29.002069 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef865fb-9f66-470d-9de7-a17de8e1f591 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:29.013412 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 450/905] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Aug 8 08:24:28 2022] GET /identity/v3/services => generated 1268 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:24:29.013412 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d591f93c-6125-49b3-8c23-891c6e382807 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:29.044583 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:29.076153 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 456/906] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Aug 8 08:24:28 2022] POST /identity/v3/services => generated 241 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:24:29.140655 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d591f93c-6125-49b3-8c23-891c6e382807 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:29.187806 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d043427-f48b-424a-ac3c-6f5399b10fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:29.188482 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d043427-f48b-424a-ac3c-6f5399b10fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:29.188822 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d043427-f48b-424a-ac3c-6f5399b10fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:29.189095 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d043427-f48b-424a-ac3c-6f5399b10fb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:29.263837 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d043427-f48b-424a-ac3c-6f5399b10fb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:29.265468 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d043427-f48b-424a-ac3c-6f5399b10fb5 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:29.291549 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:29.372789 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 451/907] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:24:29 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 08:24:29.378533 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef865fb-9f66-470d-9de7-a17de8e1f591 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:29.434841 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:29.494839 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef865fb-9f66-470d-9de7-a17de8e1f591 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:29.532231 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2180f0-9c8f-4c0f-9174-7c4583573e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:29.532879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2180f0-9c8f-4c0f-9174-7c4583573e1f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:29.533180 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2180f0-9c8f-4c0f-9174-7c4583573e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:29.533466 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2180f0-9c8f-4c0f-9174-7c4583573e1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:29.544662 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2180f0-9c8f-4c0f-9174-7c4583573e1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:29.546235 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2180f0-9c8f-4c0f-9174-7c4583573e1f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:29.561031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-bd2180f0-9c8f-4c0f-9174-7c4583573e1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:29.561031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:29.561031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:29.561031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:29.561031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:29.561031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:29.561031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:29.561031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:29.561031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:29.561031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:29.561031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:29.561031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:29.561031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:29.561031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:29.561031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:29.561031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:29.562591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:29.562591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:29.562591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:29.562591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:29.562591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:29.562591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:29.562591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:29.562591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:29.562591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:29.562591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:29.562591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:29.562591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:29.562591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:29.562591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:29.562591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:29.562591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:29.562591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:29.563474 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:29.563474 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:29.563474 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:29.563474 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 457/908] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:29.570260 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d043427-f48b-424a-ac3c-6f5399b10fb5 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:29.673740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d043427-f48b-424a-ac3c-6f5399b10fb5 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:29.696520 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:29.708762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a9a7d-4878-4dfd-9fe1-d012fb218e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:29.709397 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a9a7d-4878-4dfd-9fe1-d012fb218e10 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:29.709679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a9a7d-4878-4dfd-9fe1-d012fb218e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:29.709950 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a9a7d-4878-4dfd-9fe1-d012fb218e10 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:29.710589 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a9a7d-4878-4dfd-9fe1-d012fb218e10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:29.710890 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a9a7d-4878-4dfd-9fe1-d012fb218e10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:29.712251 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a9a7d-4878-4dfd-9fe1-d012fb218e10 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:29.726078 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 452/909] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:29.731860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2180f0-9c8f-4c0f-9174-7c4583573e1f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:29.830672 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:29.837446 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2180f0-9c8f-4c0f-9174-7c4583573e1f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:29.873029 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b66a91-019d-432c-8a21-3369368e90a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:29.873835 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b66a91-019d-432c-8a21-3369368e90a3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:29.874205 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b66a91-019d-432c-8a21-3369368e90a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:29.874470 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b66a91-019d-432c-8a21-3369368e90a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:29.881497 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b66a91-019d-432c-8a21-3369368e90a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:29.883010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b66a91-019d-432c-8a21-3369368e90a3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:29.895038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-f9b66a91-019d-432c-8a21-3369368e90a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:29.895038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:29.895038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:29.895038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:29.895038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:29.895038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:29.895038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:29.895038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:29.895038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:29.895038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:29.895038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:29.895038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:29.895038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:24:29.895038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:24:29.895038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:24:29.895989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:24:29.895989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:29.895989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:29.895989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:29.895989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:29.895989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:29.895989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:29.895989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:29.895989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:29.895989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:29.895989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:29.895989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:29.895989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:29.895989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:29.895989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:29.895989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:29.895989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:29.896896 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:24:29.896896 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:24:29.896896 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:24:29.896896 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:24:29.896896 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:24:29.896896 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:24:29.896896 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:29.896896 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:29.896896 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 458/910] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:24:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:29.902218 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a9a7d-4878-4dfd-9fe1-d012fb218e10 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:30.007097 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a9a7d-4878-4dfd-9fe1-d012fb218e10 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:30.040830 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb5154-ee60-4db1-92bb-0d33e58cda9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:30.041479 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb5154-ee60-4db1-92bb-0d33e58cda9a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:30.041773 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb5154-ee60-4db1-92bb-0d33e58cda9a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:30.042134 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb5154-ee60-4db1-92bb-0d33e58cda9a None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:30.042834 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb5154-ee60-4db1-92bb-0d33e58cda9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:30.043139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb5154-ee60-4db1-92bb-0d33e58cda9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:30.044561 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb5154-ee60-4db1-92bb-0d33e58cda9a None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:30.058718 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 453/911] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:24:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:30.065536 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b66a91-019d-432c-8a21-3369368e90a3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:30.170648 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b66a91-019d-432c-8a21-3369368e90a3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:30.207764 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a86da7-2009-49ac-825d-f33394c49ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:30.208676 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a86da7-2009-49ac-825d-f33394c49ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:30.208960 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a86da7-2009-49ac-825d-f33394c49ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:30.209262 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a86da7-2009-49ac-825d-f33394c49ea5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:30.210004 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a86da7-2009-49ac-825d-f33394c49ea5 None admin] RBAC: Adding query filter params (role.id=b4f3adb27a0048ea86dbf52cb1d2ece5, scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4, user.id=7f3422c6f9a041ceba851b71bd15d0df) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:30.210314 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a86da7-2009-49ac-825d-f33394c49ea5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:30.211769 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a86da7-2009-49ac-825d-f33394c49ea5 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:30.219042 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 459/912] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:24:30 2022] GET /identity/v3/role_assignments?user.id=7f3422c6f9a041ceba851b71bd15d0df&scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4&role.id=b4f3adb27a0048ea86dbf52cb1d2ece5 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:30.452405 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d9856-99d6-4550-bd02-f3315e83812c None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:30.452903 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d9856-99d6-4550-bd02-f3315e83812c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:30.452979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d9856-99d6-4550-bd02-f3315e83812c None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:30.454571 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 454/913] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:30.459928 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24c0ce-2e1c-4946-aa57-94dc653bdf84 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:30.460316 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24c0ce-2e1c-4946-aa57-94dc653bdf84 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:30.460481 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24c0ce-2e1c-4946-aa57-94dc653bdf84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:30.483174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-fe24c0ce-2e1c-4946-aa57-94dc653bdf84 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:30.484337 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fe24c0ce-2e1c-4946-aa57-94dc653bdf84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:30.497467 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:30.557277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 460/914] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:30.576207 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d9856-99d6-4550-bd02-f3315e83812c None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:30.622247 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:30.683633 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d9856-99d6-4550-bd02-f3315e83812c None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:30.720355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae991a6-4fdd-4568-b848-d57e8365ae10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:30.721040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae991a6-4fdd-4568-b848-d57e8365ae10 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:30.721360 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae991a6-4fdd-4568-b848-d57e8365ae10 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:30.721629 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae991a6-4fdd-4568-b848-d57e8365ae10 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:30.722210 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae991a6-4fdd-4568-b848-d57e8365ae10 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:30.723556 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae991a6-4fdd-4568-b848-d57e8365ae10 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:30.732653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-dae991a6-4fdd-4568-b848-d57e8365ae10 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 08:24:30.732653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:30.732653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 08:24:30.732653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 08:24:30.732653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 08:24:30.732653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 08:24:30.732653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 08:24:30.732653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:30.732653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 08:24:30.732653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:30.732653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:24:30.732653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:30.732653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:30.732653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:30.732653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:30.732653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:30.734141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:30.734141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:30.734141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:30.734141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:30.734141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:30.734141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:30.734141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:30.734141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 08:24:30.734141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 08:24:30.734141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 08:24:30.734141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 08:24:30.734141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:30.734141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:30.734141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:30.734141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:30.734141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:30.734141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:30.735021 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:30.735021 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:30.735021 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:30.735021 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:30.735021 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:30.735021 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:30.735021 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:30.735021 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:30.735021 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:30.735021 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:30.735021 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 08:24:30.735021 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:30.735021 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 08:24:30.735021 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:30.738547 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 455/915] 10.222.0.33 () {56 vars in 1132 bytes} [Mon Aug 8 08:24:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:30.740944 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe24c0ce-2e1c-4946-aa57-94dc653bdf84 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:30.841349 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe24c0ce-2e1c-4946-aa57-94dc653bdf84 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:30.877488 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc550f-942f-4a2f-9930-d6f87343fccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:30.878108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc550f-942f-4a2f-9930-d6f87343fccc None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:30.878386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc550f-942f-4a2f-9930-d6f87343fccc None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:30.878660 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc550f-942f-4a2f-9930-d6f87343fccc None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:30.879298 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc550f-942f-4a2f-9930-d6f87343fccc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:30.879602 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc550f-942f-4a2f-9930-d6f87343fccc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:30.880983 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc550f-942f-4a2f-9930-d6f87343fccc None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:30.886418 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 461/916] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Aug 8 08:24:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:30.892817 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae991a6-4fdd-4568-b848-d57e8365ae10 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:30.999147 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae991a6-4fdd-4568-b848-d57e8365ae10 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:31.006456 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:31.036464 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee517eb-efcb-4c5f-86fa-4e2c9dc8bd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:31.037066 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee517eb-efcb-4c5f-86fa-4e2c9dc8bd6d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:31.037370 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee517eb-efcb-4c5f-86fa-4e2c9dc8bd6d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:31.037639 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee517eb-efcb-4c5f-86fa-4e2c9dc8bd6d None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:31.038262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee517eb-efcb-4c5f-86fa-4e2c9dc8bd6d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:31.039539 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee517eb-efcb-4c5f-86fa-4e2c9dc8bd6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:31.040906 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee517eb-efcb-4c5f-86fa-4e2c9dc8bd6d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:31.050699 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 456/917] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Aug 8 08:24:30 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:31.052164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc550f-942f-4a2f-9930-d6f87343fccc None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:31.068649 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:31.153500 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95825cd-554b-46ff-9a7a-f3313e04fcde None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:31.154144 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95825cd-554b-46ff-9a7a-f3313e04fcde None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:31.154648 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95825cd-554b-46ff-9a7a-f3313e04fcde None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:31.156511 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 457/918] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:31.160265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc550f-942f-4a2f-9930-d6f87343fccc None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:31.161774 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3c563-2c4d-48ea-95d0-850b07861314 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:31.162272 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3c563-2c4d-48ea-95d0-850b07861314 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:31.162764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3c563-2c4d-48ea-95d0-850b07861314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:31.181901 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-f6f3c563-2c4d-48ea-95d0-850b07861314 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:31.183496 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f6f3c563-2c4d-48ea-95d0-850b07861314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:31.195689 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54512ee7-f195-43b1-8880-b4f7a662ecf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:31.196302 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54512ee7-f195-43b1-8880-b4f7a662ecf9 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:31.196582 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54512ee7-f195-43b1-8880-b4f7a662ecf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:31.196876 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54512ee7-f195-43b1-8880-b4f7a662ecf9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:31.197558 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54512ee7-f195-43b1-8880-b4f7a662ecf9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:31.199060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54512ee7-f195-43b1-8880-b4f7a662ecf9 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:31.258177 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 458/919] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:31.275177 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 462/920] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:24:31 2022] POST /identity/v3/endpoints => generated 347 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:24:31.283579 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3c563-2c4d-48ea-95d0-850b07861314 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:31.362309 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e4388-997e-4472-9533-f776e0616838 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:31.362651 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e4388-997e-4472-9533-f776e0616838 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:31.362864 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e4388-997e-4472-9533-f776e0616838 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:31.364578 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 463/921] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:31.370361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8a01b-fa1f-4986-82aa-eb1dd3b69423 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:31.370694 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8a01b-fa1f-4986-82aa-eb1dd3b69423 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:31.370902 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8a01b-fa1f-4986-82aa-eb1dd3b69423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:31.390986 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-54f8a01b-fa1f-4986-82aa-eb1dd3b69423 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:31.392402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-54f8a01b-fa1f-4986-82aa-eb1dd3b69423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:31.409867 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:31.414118 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3c563-2c4d-48ea-95d0-850b07861314 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:31.483139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cbdfe-d59c-44a3-9bbc-22f7b76c7adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:31.483835 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cbdfe-d59c-44a3-9bbc-22f7b76c7adf None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:31.484032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cbdfe-d59c-44a3-9bbc-22f7b76c7adf None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:31.484297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cbdfe-d59c-44a3-9bbc-22f7b76c7adf None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:31.484869 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18cbdfe-d59c-44a3-9bbc-22f7b76c7adf None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:31.486254 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18cbdfe-d59c-44a3-9bbc-22f7b76c7adf None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:31.492662 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:31.496494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-f18cbdfe-d59c-44a3-9bbc-22f7b76c7adf None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 08:24:31.496494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:31.496494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 08:24:31.496494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 08:24:31.496494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 08:24:31.496494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 08:24:31.496494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 08:24:31.496494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:31.496494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 08:24:31.496494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:31.496494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:24:31.496494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:31.496494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:31.496494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:31.496494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:31.496494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:31.496494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:31.497722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:31.497722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:31.497722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:31.497722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:31.497722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:31.497722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:31.497722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 08:24:31.497722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 08:24:31.497722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 08:24:31.497722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 08:24:31.497722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:31.497722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:31.497722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:31.497722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:31.497722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:31.497722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:31.499085 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:31.499085 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:31.499085 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:31.499085 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:31.499085 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:31.499085 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:31.499085 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:31.499085 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:31.499085 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:31.499085 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:31.499085 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 08:24:31.499085 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:31.499085 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 08:24:31.499085 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:31.501394 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 459/922] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 08:24:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:31.504239 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 464/923] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:31.507826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cbdfe-d59c-44a3-9bbc-22f7b76c7adf None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:31.526979 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f8a01b-fa1f-4986-82aa-eb1dd3b69423 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:31.616736 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cbdfe-d59c-44a3-9bbc-22f7b76c7adf None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:31.632864 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f8a01b-fa1f-4986-82aa-eb1dd3b69423 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:31.659894 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c094d4e-d5b1-43db-9490-1fd69e44e74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:31.660824 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c094d4e-d5b1-43db-9490-1fd69e44e74b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:31.661111 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c094d4e-d5b1-43db-9490-1fd69e44e74b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:31.661415 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c094d4e-d5b1-43db-9490-1fd69e44e74b None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:31.662321 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c094d4e-d5b1-43db-9490-1fd69e44e74b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:31.662626 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c094d4e-d5b1-43db-9490-1fd69e44e74b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:31.664645 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c094d4e-d5b1-43db-9490-1fd69e44e74b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:31.669430 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e92308-49a4-44c5-b536-4bb7768ba5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:31.673196 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e92308-49a4-44c5-b536-4bb7768ba5d3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:31.673336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 460/924] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 08:24:31 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:31.675419 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e92308-49a4-44c5-b536-4bb7768ba5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:31.675756 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e92308-49a4-44c5-b536-4bb7768ba5d3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:31.676388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c094d4e-d5b1-43db-9490-1fd69e44e74b None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:31.676495 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e92308-49a4-44c5-b536-4bb7768ba5d3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:31.678179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e92308-49a4-44c5-b536-4bb7768ba5d3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:31.694146 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-d2e92308-49a4-44c5-b536-4bb7768ba5d3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 08:24:31.694146 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:31.694146 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 08:24:31.694146 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 08:24:31.694146 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 08:24:31.694146 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 08:24:31.694146 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 08:24:31.694146 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:31.694146 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 08:24:31.694146 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:31.694146 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:24:31.694146 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:31.694146 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:31.694146 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:31.694146 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:31.694146 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:31.694146 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:31.695398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:31.695398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:31.695398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:31.695398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:31.695398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:31.695398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:31.695398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 08:24:31.695398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 08:24:31.695398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 08:24:31.695398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 08:24:31.695398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:31.695398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:31.695398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:31.695398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:31.695398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:31.695398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:31.696442 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:31.696442 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:31.696442 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:31.696442 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:31.696442 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:31.696442 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:31.696442 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:31.696442 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:31.696442 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:31.696442 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:31.696442 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 08:24:31.696442 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:31.696442 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 08:24:31.696442 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:31.699333 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 465/925] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Aug 8 08:24:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:24:31.705431 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e92308-49a4-44c5-b536-4bb7768ba5d3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:31.719929 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:31.775031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:31.807723 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c094d4e-d5b1-43db-9490-1fd69e44e74b None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:31.825754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e92308-49a4-44c5-b536-4bb7768ba5d3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:31.850313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a4bce-2060-49b1-98b1-c3372b331a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:31.851099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a4bce-2060-49b1-98b1-c3372b331a7c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:31.851482 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a4bce-2060-49b1-98b1-c3372b331a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:31.851856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a4bce-2060-49b1-98b1-c3372b331a7c None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:31.853214 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a4bce-2060-49b1-98b1-c3372b331a7c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:31.853632 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a4bce-2060-49b1-98b1-c3372b331a7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:31.855648 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a4bce-2060-49b1-98b1-c3372b331a7c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:31.863843 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 461/926] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 08:24:31 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:31.869303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a4bce-2060-49b1-98b1-c3372b331a7c None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:31.872235 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c9920-d299-46b6-ba56-5c5c067dce73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:31.872854 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c9920-d299-46b6-ba56-5c5c067dce73 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:31.873130 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c9920-d299-46b6-ba56-5c5c067dce73 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:31.873618 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c9920-d299-46b6-ba56-5c5c067dce73 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:31.874266 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c9920-d299-46b6-ba56-5c5c067dce73 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:31.874612 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c9920-d299-46b6-ba56-5c5c067dce73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:31.876033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c9920-d299-46b6-ba56-5c5c067dce73 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:31.881569 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 466/927] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Aug 8 08:24:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:31.887677 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c9920-d299-46b6-ba56-5c5c067dce73 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:31.970727 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:31.985186 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a4bce-2060-49b1-98b1-c3372b331a7c None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:32.022156 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c9920-d299-46b6-ba56-5c5c067dce73 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:32.022597 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb8a57-f596-4489-b20a-3a963239bd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:32.023448 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb8a57-f596-4489-b20a-3a963239bd84 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:32.024009 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb8a57-f596-4489-b20a-3a963239bd84 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:32.024471 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb8a57-f596-4489-b20a-3a963239bd84 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:32.025395 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cb8a57-f596-4489-b20a-3a963239bd84 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:32.027081 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cb8a57-f596-4489-b20a-3a963239bd84 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:32.053649 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:32.063529 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177491e-26cc-4465-86fb-f2ec60de6161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:32.064310 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177491e-26cc-4465-86fb-f2ec60de6161 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:32.064758 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177491e-26cc-4465-86fb-f2ec60de6161 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:32.065035 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177491e-26cc-4465-86fb-f2ec60de6161 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:32.065783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9177491e-26cc-4465-86fb-f2ec60de6161 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:32.066085 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9177491e-26cc-4465-86fb-f2ec60de6161 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:32.067603 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9177491e-26cc-4465-86fb-f2ec60de6161 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:32.077276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 467/928] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Aug 8 08:24:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:32.079793 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177491e-26cc-4465-86fb-f2ec60de6161 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:32.117835 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 462/929] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:24:31 2022] POST /identity/v3/endpoints => generated 346 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:24:32.210136 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177491e-26cc-4465-86fb-f2ec60de6161 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:32.257178 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2351d013-9f92-4840-aae2-34735d1fddf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:32.257860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351d013-9f92-4840-aae2-34735d1fddf4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:32.258137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351d013-9f92-4840-aae2-34735d1fddf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:32.258404 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351d013-9f92-4840-aae2-34735d1fddf4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:32.259157 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2351d013-9f92-4840-aae2-34735d1fddf4 None admin] RBAC: Adding query filter params (interface=public, service_id=1e921f2143df4fa1aa98e2f87ac53434, region_id=RegionOne) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:32.259466 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2351d013-9f92-4840-aae2-34735d1fddf4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:32.260852 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2351d013-9f92-4840-aae2-34735d1fddf4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:32.267873 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 468/930] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Aug 8 08:24:32 2022] GET /identity/v3/endpoints?service_id=1e921f2143df4fa1aa98e2f87ac53434&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:32.273322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb8a57-f596-4489-b20a-3a963239bd84 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:32.337172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805ec2b-c80d-4cbf-89a8-ca4fa5b7159e None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:32.337557 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805ec2b-c80d-4cbf-89a8-ca4fa5b7159e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:32.337923 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805ec2b-c80d-4cbf-89a8-ca4fa5b7159e None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:32.344231 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 469/931] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:32.345860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb2e1c-84ba-4394-a43e-c20dfb200030 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:32.346245 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb2e1c-84ba-4394-a43e-c20dfb200030 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:32.346614 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb2e1c-84ba-4394-a43e-c20dfb200030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:32.374728 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b9bb2e1c-84ba-4394-a43e-c20dfb200030 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:32.376053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b9bb2e1c-84ba-4394-a43e-c20dfb200030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:32.399835 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb8a57-f596-4489-b20a-3a963239bd84 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:32.438446 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:32.441395 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a66dbf-7f2f-46b1-80f8-c250ebb7b2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:32.442015 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a66dbf-7f2f-46b1-80f8-c250ebb7b2de None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:32.442301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a66dbf-7f2f-46b1-80f8-c250ebb7b2de None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:32.442571 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a66dbf-7f2f-46b1-80f8-c250ebb7b2de None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:32.443183 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a66dbf-7f2f-46b1-80f8-c250ebb7b2de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:32.444537 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a66dbf-7f2f-46b1-80f8-c250ebb7b2de None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:32.451004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 463/932] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Aug 8 08:24:32 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:24:32.475019 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 470/933] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 08:24:32.501249 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a66dbf-7f2f-46b1-80f8-c250ebb7b2de None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:32.513827 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:32.633536 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a66dbf-7f2f-46b1-80f8-c250ebb7b2de None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:32.674645 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e725141-7a28-49a4-8969-a98f7980fa6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:32.675278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e725141-7a28-49a4-8969-a98f7980fa6e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:32.675559 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e725141-7a28-49a4-8969-a98f7980fa6e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:32.675828 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e725141-7a28-49a4-8969-a98f7980fa6e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:32.686335 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e725141-7a28-49a4-8969-a98f7980fa6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:24:32.692265 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-5e725141-7a28-49a4-8969-a98f7980fa6e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 08:24:32.692265 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:32.692265 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:32.692265 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:32.692265 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:32.692265 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:32.692265 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:32.692265 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:32.692265 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:32.692265 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:32.692265 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:32.692265 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:32.692265 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:24:32.692265 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:24:32.692265 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:24:32.692265 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:24:32.693206 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:24:32.693206 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:24:32.693206 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:24:32.693206 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:24:32.693206 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:32.693206 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:32.693206 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:32.693206 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:32.693206 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:32.693206 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:32.693206 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:32.693206 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:32.693206 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:32.693206 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:32.693206 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:32.693206 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:32.694060 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:32.694060 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:32.694060 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:32.694060 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:32.694060 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:24:32.694060 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:24:32.694060 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:24:32.694060 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:24:32.694060 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:24:32.694060 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:24:32.694060 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 08:24:32.694060 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:32.694060 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 464/934] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Aug 8 08:24:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:32.701170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb2e1c-84ba-4394-a43e-c20dfb200030 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:32.824874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb2e1c-84ba-4394-a43e-c20dfb200030 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:32.881934 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f25d1-ab14-4eb9-95c4-b9f2c74df743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:32.883066 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f25d1-ab14-4eb9-95c4-b9f2c74df743 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:32.883604 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f25d1-ab14-4eb9-95c4-b9f2c74df743 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:32.884082 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f25d1-ab14-4eb9-95c4-b9f2c74df743 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:32.884980 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f25d1-ab14-4eb9-95c4-b9f2c74df743 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:32.885523 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f25d1-ab14-4eb9-95c4-b9f2c74df743 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:32.887114 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f25d1-ab14-4eb9-95c4-b9f2c74df743 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:32.897093 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 471/935] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Aug 8 08:24:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:32.902587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e725141-7a28-49a4-8969-a98f7980fa6e None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:32.975062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:33.008980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e725141-7a28-49a4-8969-a98f7980fa6e None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:33.044360 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadfe9d2-2081-43aa-83a9-965348a8b432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:33.044990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadfe9d2-2081-43aa-83a9-965348a8b432 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:33.045301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadfe9d2-2081-43aa-83a9-965348a8b432 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:33.045692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadfe9d2-2081-43aa-83a9-965348a8b432 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:33.056162 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadfe9d2-2081-43aa-83a9-965348a8b432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:33.057701 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadfe9d2-2081-43aa-83a9-965348a8b432 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:33.073283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-aadfe9d2-2081-43aa-83a9-965348a8b432 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:33.073283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:33.073283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:33.073283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:33.073283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:33.073283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:33.073283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:33.073283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:33.073283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:33.073283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:33.073283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:33.073283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:33.073283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:33.073283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:33.073283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:33.073283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:33.074311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:33.074311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:33.074311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:33.074311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:33.074311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:33.074311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:33.074311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:33.074311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:33.074311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:33.074311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:33.074311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:33.074311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:33.074311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:33.074311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:33.074311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:33.074311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:33.074311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:33.075335 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:33.075335 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:33.075335 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:33.075335 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 465/936] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:33.081046 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f25d1-ab14-4eb9-95c4-b9f2c74df743 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:33.126242 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:33.184966 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f25d1-ab14-4eb9-95c4-b9f2c74df743 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:33.220715 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee650d-a2dc-4449-a73f-1dbc5705c9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:33.221646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee650d-a2dc-4449-a73f-1dbc5705c9fb None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:33.222179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee650d-a2dc-4449-a73f-1dbc5705c9fb None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:33.222642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee650d-a2dc-4449-a73f-1dbc5705c9fb None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:33.223451 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee650d-a2dc-4449-a73f-1dbc5705c9fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:33.223930 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee650d-a2dc-4449-a73f-1dbc5705c9fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:33.225485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee650d-a2dc-4449-a73f-1dbc5705c9fb None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:33.238200 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 472/937] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:33.244441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadfe9d2-2081-43aa-83a9-965348a8b432 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:33.351283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadfe9d2-2081-43aa-83a9-965348a8b432 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:33.372563 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:33.387115 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a5afd-f734-4633-bf21-dcd0ef3e3be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:33.388440 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a5afd-f734-4633-bf21-dcd0ef3e3be4 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:33.388440 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a5afd-f734-4633-bf21-dcd0ef3e3be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:33.388614 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a5afd-f734-4633-bf21-dcd0ef3e3be4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:33.397111 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241a5afd-f734-4633-bf21-dcd0ef3e3be4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:33.398696 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241a5afd-f734-4633-bf21-dcd0ef3e3be4 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:33.411097 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-241a5afd-f734-4633-bf21-dcd0ef3e3be4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 08:24:33.411097 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:33.411097 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:33.411097 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:33.411097 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:33.411097 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:33.411097 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:33.411097 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:33.411097 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:33.411097 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:33.411097 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:33.411097 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:33.411097 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:24:33.411097 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:24:33.411097 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:24:33.411097 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:24:33.412262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:33.412262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:33.412262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:24:33.412262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:33.412262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:24:33.412262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:33.412262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:33.412262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:33.412262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:33.412262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:33.412262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:33.412262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:33.412262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:33.412262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:33.412262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:33.412262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:33.412262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:33.412262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:33.413454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:33.413454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:33.413454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:24:33.413454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:24:33.413454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:24:33.413454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:24:33.413454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:24:33.413454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:24:33.413454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 08:24:33.413454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:33.414760 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 466/938] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Aug 8 08:24:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:33.420797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee650d-a2dc-4449-a73f-1dbc5705c9fb None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:33.493980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc19a16-0a2a-4a0c-a752-b4ac5ac8b31f None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:33.494178 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc19a16-0a2a-4a0c-a752-b4ac5ac8b31f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:33.494442 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc19a16-0a2a-4a0c-a752-b4ac5ac8b31f None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:33.496010 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 467/939] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:33.500856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f6116-f031-4161-8b31-04e65c6d25d9 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:33.501137 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f6116-f031-4161-8b31-04e65c6d25d9 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:33.501452 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f6116-f031-4161-8b31-04e65c6d25d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:33.524294 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-dd2f6116-f031-4161-8b31-04e65c6d25d9 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:33.525661 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-dd2f6116-f031-4161-8b31-04e65c6d25d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:33.531236 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee650d-a2dc-4449-a73f-1dbc5705c9fb None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:33.533001 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:33.583675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4f01a-1635-4b87-abee-908a1a2f802a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:33.584684 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4f01a-1635-4b87-abee-908a1a2f802a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:33.585194 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4f01a-1635-4b87-abee-908a1a2f802a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:33.586204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4f01a-1635-4b87-abee-908a1a2f802a None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:33.587419 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b4f01a-1635-4b87-abee-908a1a2f802a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:33.587946 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b4f01a-1635-4b87-abee-908a1a2f802a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:33.590501 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b4f01a-1635-4b87-abee-908a1a2f802a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:33.609650 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 468/940] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 08:24:33.612216 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 473/941] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Aug 8 08:24:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:33.617982 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f6116-f031-4161-8b31-04e65c6d25d9 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:33.623346 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4f01a-1635-4b87-abee-908a1a2f802a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:33.738413 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4f01a-1635-4b87-abee-908a1a2f802a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:33.745055 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f6116-f031-4161-8b31-04e65c6d25d9 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:33.769246 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:33.783888 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0be00-3b43-44ae-8e52-34890b14d0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:33.784998 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0be00-3b43-44ae-8e52-34890b14d0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:33.785874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0be00-3b43-44ae-8e52-34890b14d0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:33.786607 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0be00-3b43-44ae-8e52-34890b14d0f2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:33.787394 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c0be00-3b43-44ae-8e52-34890b14d0f2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:33.788956 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c0be00-3b43-44ae-8e52-34890b14d0f2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:33.790485 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1627a643-df78-4739-88b0-4a7ff95c04f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:33.791381 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627a643-df78-4739-88b0-4a7ff95c04f2 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:33.791666 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627a643-df78-4739-88b0-4a7ff95c04f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:33.791946 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627a643-df78-4739-88b0-4a7ff95c04f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:33.800260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-d8c0be00-3b43-44ae-8e52-34890b14d0f2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 08:24:33.800260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:33.800260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 08:24:33.800260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 08:24:33.800260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 08:24:33.800260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 08:24:33.800260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 08:24:33.800260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:33.800260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 08:24:33.800260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:33.800260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:24:33.800260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:33.800260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:33.800260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:33.800260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:33.800260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:33.800260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:33.806511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:33.806511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:33.806511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:33.806511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:33.806511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:33.806511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:33.806511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 08:24:33.806511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 08:24:33.806511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 08:24:33.806511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 08:24:33.806511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:33.806511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:33.806511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:33.806511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:33.806511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:33.806511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:33.807350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:33.807350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:33.807350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:33.807350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:33.807350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:33.807350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:33.807350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:33.807350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:33.807350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:33.807350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:33.807350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 08:24:33.807350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:33.807350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 08:24:33.807350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:33.807350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 474/942] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:24:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:33.809318 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0be00-3b43-44ae-8e52-34890b14d0f2 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:33.810003 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1627a643-df78-4739-88b0-4a7ff95c04f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:33.812197 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1627a643-df78-4739-88b0-4a7ff95c04f2 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:33.832778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-1627a643-df78-4739-88b0-4a7ff95c04f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:33.832778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:33.832778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:33.832778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:33.832778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:33.832778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:33.832778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:33.832778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:33.832778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:33.832778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:33.832778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:33.832778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:33.832778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:33.832778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:33.832778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:33.832778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:33.834292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:33.834292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:33.834292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:33.834292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:33.834292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:33.834292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:33.834292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:33.834292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:33.834292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:33.834292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:33.834292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:33.834292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:33.834292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:33.834292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:33.834292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:33.834292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:33.834292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:33.835237 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:33.835237 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:33.835237 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:33.840290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 469/943] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:33.841640 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1627a643-df78-4739-88b0-4a7ff95c04f2 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:33.846268 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:33.925908 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0be00-3b43-44ae-8e52-34890b14d0f2 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:33.963603 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a8b6c-3216-47d4-9cdd-3fd3a4257c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:33.963964 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a8b6c-3216-47d4-9cdd-3fd3a4257c48 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:33.964238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a8b6c-3216-47d4-9cdd-3fd3a4257c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:33.964510 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a8b6c-3216-47d4-9cdd-3fd3a4257c48 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:33.965157 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1a8b6c-3216-47d4-9cdd-3fd3a4257c48 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:33.965531 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1a8b6c-3216-47d4-9cdd-3fd3a4257c48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:33.966827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1a8b6c-3216-47d4-9cdd-3fd3a4257c48 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:33.973180 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 475/944] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Aug 8 08:24:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:33.977389 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a8b6c-3216-47d4-9cdd-3fd3a4257c48 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:33.978685 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1627a643-df78-4739-88b0-4a7ff95c04f2 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:34.016563 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2621c529-e643-47ca-81eb-8b5b77b20c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:34.017091 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621c529-e643-47ca-81eb-8b5b77b20c93 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:34.017412 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621c529-e643-47ca-81eb-8b5b77b20c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:34.017681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621c529-e643-47ca-81eb-8b5b77b20c93 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:34.018322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2621c529-e643-47ca-81eb-8b5b77b20c93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:34.018624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2621c529-e643-47ca-81eb-8b5b77b20c93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:34.020371 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2621c529-e643-47ca-81eb-8b5b77b20c93 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:34.033133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 470/945] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:34.041525 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2621c529-e643-47ca-81eb-8b5b77b20c93 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:34.066862 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:34.094534 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:34.110920 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a8b6c-3216-47d4-9cdd-3fd3a4257c48 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:34.154315 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d85824-90ee-45c7-8e70-81cd17894d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:34.155260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d85824-90ee-45c7-8e70-81cd17894d56 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:34.155697 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d85824-90ee-45c7-8e70-81cd17894d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:34.156276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d85824-90ee-45c7-8e70-81cd17894d56 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:34.157391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d85824-90ee-45c7-8e70-81cd17894d56 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:34.157980 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d85824-90ee-45c7-8e70-81cd17894d56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:34.159598 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d85824-90ee-45c7-8e70-81cd17894d56 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:34.160149 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2621c529-e643-47ca-81eb-8b5b77b20c93 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:34.173812 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 476/946] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Aug 8 08:24:33 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:34.205122 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded30da6-2888-4798-b0b5-f38fd3016f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:34.205783 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded30da6-2888-4798-b0b5-f38fd3016f61 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:34.206059 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded30da6-2888-4798-b0b5-f38fd3016f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:34.206374 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded30da6-2888-4798-b0b5-f38fd3016f61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:34.214535 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded30da6-2888-4798-b0b5-f38fd3016f61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:34.216033 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded30da6-2888-4798-b0b5-f38fd3016f61 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:34.229552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-ded30da6-2888-4798-b0b5-f38fd3016f61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:34.229552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:34.229552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:34.229552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:34.229552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:34.229552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:34.229552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:34.229552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:34.229552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:34.229552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:34.229552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:34.229552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:34.229552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:24:34.229552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:24:34.229552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:24:34.230530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:24:34.230530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:34.230530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:34.230530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:34.230530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:34.230530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:34.230530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:34.230530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:34.230530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:34.230530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:34.230530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:34.230530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:34.230530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:34.230530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:34.230530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:34.230530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:34.230530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:34.231438 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:24:34.231438 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:24:34.231438 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:24:34.231438 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:24:34.231438 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:24:34.231438 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:24:34.231438 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:34.231438 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:34.231438 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 471/947] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:24:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:34.238343 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d85824-90ee-45c7-8e70-81cd17894d56 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:34.350474 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d85824-90ee-45c7-8e70-81cd17894d56 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:34.388045 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ef29d-b310-4085-9e8e-cc1d7fcdcca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:34.388812 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ef29d-b310-4085-9e8e-cc1d7fcdcca4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:34.390173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ef29d-b310-4085-9e8e-cc1d7fcdcca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:34.390540 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ef29d-b310-4085-9e8e-cc1d7fcdcca4 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:34.391513 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ef29d-b310-4085-9e8e-cc1d7fcdcca4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:34.391834 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b51eb1-18eb-4989-a7ac-3cebdd655d40 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:34.392006 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ef29d-b310-4085-9e8e-cc1d7fcdcca4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:34.392409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b51eb1-18eb-4989-a7ac-3cebdd655d40 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:34.392833 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b51eb1-18eb-4989-a7ac-3cebdd655d40 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:34.393786 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ef29d-b310-4085-9e8e-cc1d7fcdcca4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:34.394808 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 472/948] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 08:24:34.399922 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d9f6c-1a78-4f33-977d-2ef8e68f2592 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:34.400373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d9f6c-1a78-4f33-977d-2ef8e68f2592 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:34.400787 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d9f6c-1a78-4f33-977d-2ef8e68f2592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:34.409311 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 477/949] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:24:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:34.413851 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ef29d-b310-4085-9e8e-cc1d7fcdcca4 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:34.429749 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:34.439913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-282d9f6c-1a78-4f33-977d-2ef8e68f2592 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:34.441837 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-282d9f6c-1a78-4f33-977d-2ef8e68f2592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:34.526436 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ef29d-b310-4085-9e8e-cc1d7fcdcca4 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:34.536180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 473/950] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:34.558824 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d9f6c-1a78-4f33-977d-2ef8e68f2592 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:34.570288 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97147b-830c-4175-a5fa-fd34a66a67e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:34.571612 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97147b-830c-4175-a5fa-fd34a66a67e4 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:34.572047 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97147b-830c-4175-a5fa-fd34a66a67e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:34.572729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97147b-830c-4175-a5fa-fd34a66a67e4 None admin] PATH_INFO: `/v3/projects/71e5e82c9f454cdd91dfeac5a57968c4/users/7f3422c6f9a041ceba851b71bd15d0df/roles/b4f3adb27a0048ea86dbf52cb1d2ece5` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:34.574233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:34.577825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d97147b-830c-4175-a5fa-fd34a66a67e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=71e5e82c9f454cdd91dfeac5a57968c4, user_id=7f3422c6f9a041ceba851b71bd15d0df, role_id=b4f3adb27a0048ea86dbf52cb1d2ece5)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:34.579883 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d97147b-830c-4175-a5fa-fd34a66a67e4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:34.664900 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d9f6c-1a78-4f33-977d-2ef8e68f2592 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:34.677802 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 478/951] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 08:24:34 2022] PUT /identity/v3/projects/71e5e82c9f454cdd91dfeac5a57968c4/users/7f3422c6f9a041ceba851b71bd15d0df/roles/b4f3adb27a0048ea86dbf52cb1d2ece5 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:24:34.711283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716bd47-bef0-4c24-96d3-f89fd114ffea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:34.711907 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716bd47-bef0-4c24-96d3-f89fd114ffea None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:34.712182 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716bd47-bef0-4c24-96d3-f89fd114ffea None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:34.712450 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716bd47-bef0-4c24-96d3-f89fd114ffea None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:34.713181 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9716bd47-bef0-4c24-96d3-f89fd114ffea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:34.714716 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9716bd47-bef0-4c24-96d3-f89fd114ffea None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:34.718207 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: WARNING keystone.server.flask.common [None req-9716bd47-bef0-4c24-96d3-f89fd114ffea None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 08 08:24:34.747071 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc280831-8482-4573-9df5-656ca7be0b79 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:34.747606 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc280831-8482-4573-9df5-656ca7be0b79 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:34.747606 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc280831-8482-4573-9df5-656ca7be0b79 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:34.749464 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 479/952] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 08:24:34.755296 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067b5f1-1919-4439-883a-c0fb65bae712 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:34.755440 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067b5f1-1919-4439-883a-c0fb65bae712 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:34.755730 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067b5f1-1919-4439-883a-c0fb65bae712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:34.771839 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.sql.core [None req-9716bd47-bef0-4c24-96d3-f89fd114ffea None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 08:24:34.771839 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 08 08:24:34.771839 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [parameters: {'id': '59d4bf7fe64843fb97fd2911443f1520', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 08 08:24:34.771839 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80262) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 08:24:34.778103 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a067b5f1-1919-4439-883a-c0fb65bae712 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:34.779722 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a067b5f1-1919-4439-883a-c0fb65bae712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:34.788702 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-9716bd47-bef0-4c24-96d3-f89fd114ffea None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 59d4bf7fe64843fb97fd2911443f1520.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 59d4bf7fe64843fb97fd2911443f1520. Aug 08 08:24:34.788702 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:34.788702 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 08:24:34.788702 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 08:24:34.788702 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 08:24:34.788702 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 08:24:34.788702 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 08:24:34.788702 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 08:24:34.788702 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 08:24:34.788702 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application conn.query(q) Aug 08 08:24:34.788702 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 08:24:34.788702 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 08:24:34.788702 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 08:24:34.789877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application result.read() Aug 08 08:24:34.789877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 08:24:34.789877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 08:24:34.789877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 08:24:34.789877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 08:24:34.789877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 08:24:34.789877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 08:24:34.789877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 08:24:34.789877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 08:24:34.789877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 08:24:34.789877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:34.789877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 08:24:34.789877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:34.789877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:34.789877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 08:24:34.789877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 08:24:34.789877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 08 08:24:34.789877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 08 08:24:34.790874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 08:24:34.790874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application next(self.gen) Aug 08 08:24:34.790874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 08:24:34.790874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application yield resource Aug 08 08:24:34.790874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 08:24:34.790874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application next(self.gen) Aug 08 08:24:34.790874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 08:24:34.790874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 08:24:34.790874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 08:24:34.790874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 08:24:34.790874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 08:24:34.790874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise self.value Aug 08 08:24:34.790874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 08:24:34.790874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 08:24:34.790874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 08:24:34.790874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application session.commit() Aug 08 08:24:34.790874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 08:24:34.791763 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 08:24:34.791763 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 08:24:34.791763 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 08:24:34.791763 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 08:24:34.791763 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self.session.flush() Aug 08 08:24:34.791763 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 08:24:34.791763 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 08:24:34.791763 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 08:24:34.791763 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 08:24:34.791763 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 08:24:34.791763 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application compat.raise_( Aug 08 08:24:34.791763 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 08:24:34.791763 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception Aug 08 08:24:34.791763 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 08:24:34.791763 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 08:24:34.791763 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 08:24:34.791763 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 08:24:34.792656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 08:24:34.792656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 08:24:34.792656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 08:24:34.792656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 08:24:34.792656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 08 08:24:34.792656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 08 08:24:34.792656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 08:24:34.792656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 08:24:34.792656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 08:24:34.792656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 08:24:34.792656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 08:24:34.792656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 08:24:34.792656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 08:24:34.792656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 08:24:34.792656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 08:24:34.794720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 08:24:34.794720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 08:24:34.794720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception Aug 08 08:24:34.794720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 08:24:34.794720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 08:24:34.794720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 08:24:34.794720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 08:24:34.794720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 08:24:34.794720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 08:24:34.794720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 08:24:34.794720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application conn.query(q) Aug 08 08:24:34.794720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 08:24:34.794720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 08:24:34.794720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 08:24:34.794720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application result.read() Aug 08 08:24:34.794720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 08:24:34.794720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 08:24:34.797542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 08:24:34.797542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 08:24:34.797542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 08:24:34.797542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 08:24:34.797542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 08:24:34.797542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 08:24:34.797542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 08:24:34.797542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 08 08:24:34.797542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [parameters: {'id': '59d4bf7fe64843fb97fd2911443f1520', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 08 08:24:34.797542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 08:24:34.797542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:34.797542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:24:34.797542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:34.797542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:34.797542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 08 08:24:34.799584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 08 08:24:34.799584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 08:24:34.799584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 08:24:34.799584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 08 08:24:34.799584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:34.799584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:24:34.799584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:34.799584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:34.799584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:34.799584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:34.799584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:34.799584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:34.799584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:34.799584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:34.799584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:34.799584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:34.799584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:34.804096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:34.804096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 08 08:24:34.804096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 08 08:24:34.804096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:34.804096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:34.804096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 08 08:24:34.804096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 08 08:24:34.804096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 59d4bf7fe64843fb97fd2911443f1520. Aug 08 08:24:34.804096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:34.804096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 474/953] 10.222.0.33 () {60 vars in 1140 bytes} [Mon Aug 8 08:24:34 2022] POST /identity/v3/projects => generated 262 bytes in 232 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:24:34.804096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716bd47-bef0-4c24-96d3-f89fd114ffea None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:34.878946 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:34.901808 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 480/954] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 08:24:34.922405 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716bd47-bef0-4c24-96d3-f89fd114ffea None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:34.925137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067b5f1-1919-4439-883a-c0fb65bae712 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:34.933538 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:34.966670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f999f01-903d-4fc9-aa9c-d58a220db186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:34.967305 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f999f01-903d-4fc9-aa9c-d58a220db186 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:34.968169 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f999f01-903d-4fc9-aa9c-d58a220db186 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:34.968966 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f999f01-903d-4fc9-aa9c-d58a220db186 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:34.978038 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f999f01-903d-4fc9-aa9c-d58a220db186 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:34.980210 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f999f01-903d-4fc9-aa9c-d58a220db186 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:34.995927 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-6f999f01-903d-4fc9-aa9c-d58a220db186 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:34.995927 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:34.995927 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:34.995927 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:34.995927 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:34.995927 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:34.995927 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:34.995927 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:34.995927 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:34.995927 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:34.995927 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:34.995927 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:34.995927 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:24:34.995927 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:24:34.995927 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:24:34.997052 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:24:34.997052 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:34.997052 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:34.997052 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:34.997052 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:34.997052 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:34.997052 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:34.997052 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:34.997052 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:34.997052 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:34.997052 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:34.997052 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:34.997052 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:34.997052 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:34.997052 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:34.997052 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:34.997052 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:34.998238 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:24:34.998238 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:24:34.998238 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:24:34.998238 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:24:34.998238 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:24:34.998238 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:24:34.998238 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:34.998238 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:35.001236 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 475/955] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:24:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:35.005095 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f999f01-903d-4fc9-aa9c-d58a220db186 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:35.038587 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067b5f1-1919-4439-883a-c0fb65bae712 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:35.076539 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac859ce3-d13e-4b62-9741-01558884e90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:35.077955 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac859ce3-d13e-4b62-9741-01558884e90f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:35.078331 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac859ce3-d13e-4b62-9741-01558884e90f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:35.078754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac859ce3-d13e-4b62-9741-01558884e90f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:35.079637 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac859ce3-d13e-4b62-9741-01558884e90f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:35.081243 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac859ce3-d13e-4b62-9741-01558884e90f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:35.091193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-ac859ce3-d13e-4b62-9741-01558884e90f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 08:24:35.091193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:35.091193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 08:24:35.091193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 08:24:35.091193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 08:24:35.091193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 08:24:35.091193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 08:24:35.091193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:35.091193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 08:24:35.091193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:35.091193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:24:35.091193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:35.091193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:35.091193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:35.091193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:35.091193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:35.091193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:35.092601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:35.092601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:35.092601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:35.092601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:35.092601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:35.092601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:35.092601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 08:24:35.092601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 08:24:35.092601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 08:24:35.092601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 08:24:35.092601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:35.092601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:35.092601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:35.092601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:35.092601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:35.092601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:35.095064 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:35.095064 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:35.095064 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:35.095064 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:35.095064 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:35.095064 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:35.095064 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:35.095064 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:35.095064 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:35.095064 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:35.095064 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 08:24:35.095064 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:35.095064 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 08:24:35.095064 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:35.096169 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 481/956] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Aug 8 08:24:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:24:35.103564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac859ce3-d13e-4b62-9741-01558884e90f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:35.126725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f999f01-903d-4fc9-aa9c-d58a220db186 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:35.160695 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:35.169714 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56413463-297e-4c7a-bc09-f1b14bb885c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:35.170325 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56413463-297e-4c7a-bc09-f1b14bb885c9 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:35.170754 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56413463-297e-4c7a-bc09-f1b14bb885c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:35.171026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56413463-297e-4c7a-bc09-f1b14bb885c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:35.171713 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56413463-297e-4c7a-bc09-f1b14bb885c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:35.172141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56413463-297e-4c7a-bc09-f1b14bb885c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:35.175039 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56413463-297e-4c7a-bc09-f1b14bb885c9 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:35.179762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:35.209176 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 476/957] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Aug 8 08:24:35 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:35.224650 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac859ce3-d13e-4b62-9741-01558884e90f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:35.262616 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e8ac4-c40f-4296-a3cd-759b8d65e4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:35.263257 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e8ac4-c40f-4296-a3cd-759b8d65e4e0 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:35.263538 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e8ac4-c40f-4296-a3cd-759b8d65e4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:35.263812 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e8ac4-c40f-4296-a3cd-759b8d65e4e0 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:35.264538 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e8ac4-c40f-4296-a3cd-759b8d65e4e0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:35.264871 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e8ac4-c40f-4296-a3cd-759b8d65e4e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:35.266470 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e8ac4-c40f-4296-a3cd-759b8d65e4e0 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:35.273386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 482/958] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Aug 8 08:24:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:35.282361 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56413463-297e-4c7a-bc09-f1b14bb885c9 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:35.397893 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56413463-297e-4c7a-bc09-f1b14bb885c9 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:35.444041 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae7f73-b2db-4dce-858b-28c75886c8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:35.444579 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae7f73-b2db-4dce-858b-28c75886c8d4 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:35.444853 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae7f73-b2db-4dce-858b-28c75886c8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:35.445120 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae7f73-b2db-4dce-858b-28c75886c8d4 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:35.445795 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ae7f73-b2db-4dce-858b-28c75886c8d4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:35.446094 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ae7f73-b2db-4dce-858b-28c75886c8d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:35.447451 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ae7f73-b2db-4dce-858b-28c75886c8d4 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:35.453606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 477/959] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Aug 8 08:24:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:35.459476 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e8ac4-c40f-4296-a3cd-759b8d65e4e0 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:35.561929 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e8ac4-c40f-4296-a3cd-759b8d65e4e0 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:35.593363 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:35.597020 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e5b1f-d74a-4e30-8cc4-86f3ff007ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:35.597710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e5b1f-d74a-4e30-8cc4-86f3ff007ca4 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:35.597990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e5b1f-d74a-4e30-8cc4-86f3ff007ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:35.598264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e5b1f-d74a-4e30-8cc4-86f3ff007ca4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:35.598927 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8e5b1f-d74a-4e30-8cc4-86f3ff007ca4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:35.600275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8e5b1f-d74a-4e30-8cc4-86f3ff007ca4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:35.687257 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 483/960] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:24:35 2022] POST /identity/v3/endpoints => generated 349 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:24:36.288489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24905c21-9890-42f5-9e91-ef0450c91a38 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:36.289047 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24905c21-9890-42f5-9e91-ef0450c91a38 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:36.289047 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24905c21-9890-42f5-9e91-ef0450c91a38 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:36.290394 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 478/961] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:36.296203 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e6f94-8584-4f4c-88a1-47205ca55b33 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:36.296623 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e6f94-8584-4f4c-88a1-47205ca55b33 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:36.296817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e6f94-8584-4f4c-88a1-47205ca55b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:36.315539 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-679e6f94-8584-4f4c-88a1-47205ca55b33 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:36.316717 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-679e6f94-8584-4f4c-88a1-47205ca55b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:36.392391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 484/962] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:36.415396 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24905c21-9890-42f5-9e91-ef0450c91a38 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:36.475065 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:36.525410 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24905c21-9890-42f5-9e91-ef0450c91a38 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:36.561013 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073cb52-9145-4266-954d-64852ed25369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:36.562123 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073cb52-9145-4266-954d-64852ed25369 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:36.562400 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073cb52-9145-4266-954d-64852ed25369 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:36.562669 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073cb52-9145-4266-954d-64852ed25369 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:36.563324 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d073cb52-9145-4266-954d-64852ed25369 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:36.564675 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d073cb52-9145-4266-954d-64852ed25369 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:36.664305 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 479/963] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Aug 8 08:24:36 2022] POST /identity/v3/services => generated 242 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:24:36.808052 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81e418-cef3-4af5-b07d-e72f97a7f049 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:36.808408 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81e418-cef3-4af5-b07d-e72f97a7f049 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:36.808666 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81e418-cef3-4af5-b07d-e72f97a7f049 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:36.810347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 485/964] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:36.816587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e417e21-014a-45d1-8bef-047a01972909 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:36.816826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e417e21-014a-45d1-8bef-047a01972909 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:36.817088 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e417e21-014a-45d1-8bef-047a01972909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:36.836490 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-6e417e21-014a-45d1-8bef-047a01972909 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:36.838146 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6e417e21-014a-45d1-8bef-047a01972909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:36.921498 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 480/965] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:36 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:36.941784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81e418-cef3-4af5-b07d-e72f97a7f049 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:37.050161 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81e418-cef3-4af5-b07d-e72f97a7f049 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:37.064458 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:37.086264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e331c88-8f95-4284-8642-68a8066babf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:37.086821 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e331c88-8f95-4284-8642-68a8066babf9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:37.087096 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e331c88-8f95-4284-8642-68a8066babf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:37.087365 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e331c88-8f95-4284-8642-68a8066babf9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:37.097573 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e331c88-8f95-4284-8642-68a8066babf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:24:37.103026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-4e331c88-8f95-4284-8642-68a8066babf9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 08:24:37.103026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:37.103026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:37.103026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:37.103026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:37.103026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:37.103026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:37.103026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:37.103026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:37.103026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:37.103026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:37.103026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:37.103026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:24:37.103026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:24:37.103026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:24:37.103026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:24:37.103965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:24:37.103965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:24:37.103965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:24:37.103965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:24:37.103965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:37.103965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:37.103965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:37.103965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:37.103965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:37.103965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:37.103965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:37.103965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:37.103965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:37.103965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:37.103965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:37.103965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:37.104878 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:37.104878 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:37.104878 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:37.104878 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:37.104878 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:24:37.104878 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:24:37.104878 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:24:37.104878 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:24:37.104878 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:24:37.104878 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:24:37.104878 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 08:24:37.104878 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:37.104878 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 486/966] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Aug 8 08:24:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:37.111083 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e417e21-014a-45d1-8bef-047a01972909 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:37.151691 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:37.220796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e417e21-014a-45d1-8bef-047a01972909 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:37.257074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8137f39-6152-4729-a2e1-04771e8898a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:37.257738 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8137f39-6152-4729-a2e1-04771e8898a2 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:37.258018 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8137f39-6152-4729-a2e1-04771e8898a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:37.258293 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8137f39-6152-4729-a2e1-04771e8898a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:37.258958 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8137f39-6152-4729-a2e1-04771e8898a2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:37.259260 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8137f39-6152-4729-a2e1-04771e8898a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:37.260618 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8137f39-6152-4729-a2e1-04771e8898a2 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:37.270218 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 481/967] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Aug 8 08:24:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:37.275911 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e331c88-8f95-4284-8642-68a8066babf9 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:37.382209 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e331c88-8f95-4284-8642-68a8066babf9 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:37.420390 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d5168-5ee6-4f92-9fae-e0222ba8c6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:37.421617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d5168-5ee6-4f92-9fae-e0222ba8c6f8 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:37.421897 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d5168-5ee6-4f92-9fae-e0222ba8c6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:37.422163 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d5168-5ee6-4f92-9fae-e0222ba8c6f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:37.432979 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449d5168-5ee6-4f92-9fae-e0222ba8c6f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:37.434463 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449d5168-5ee6-4f92-9fae-e0222ba8c6f8 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:37.436523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8d821-dc9d-4673-8fa8-7fd21f0c5cc5 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:37.437005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8d821-dc9d-4673-8fa8-7fd21f0c5cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:37.437097 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8d821-dc9d-4673-8fa8-7fd21f0c5cc5 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:37.438581 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 482/968] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:37.443683 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a065a-8e85-4cc7-a1d4-bcd599a958d1 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:37.444082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a065a-8e85-4cc7-a1d4-bcd599a958d1 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:37.444460 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a065a-8e85-4cc7-a1d4-bcd599a958d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:37.455056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-449d5168-5ee6-4f92-9fae-e0222ba8c6f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:37.455056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:37.455056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:37.455056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:37.455056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:37.455056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:37.455056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:37.455056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:37.455056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:37.455056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:37.455056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:37.455056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:37.455056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:37.455056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:37.455056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:37.455056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:37.456211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:37.456211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:37.456211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:37.456211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:37.456211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:37.456211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:37.456211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:37.456211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:37.456211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:37.456211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:37.456211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:37.456211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:37.456211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:37.456211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:37.456211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:37.456211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:37.456211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:37.457333 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:37.457333 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:37.457333 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:37.459429 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 487/969] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:37.466055 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-541a065a-8e85-4cc7-a1d4-bcd599a958d1 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:37.467251 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-541a065a-8e85-4cc7-a1d4-bcd599a958d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:37.471303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d5168-5ee6-4f92-9fae-e0222ba8c6f8 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:37.506001 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:37.549292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 483/970] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:37 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 08:24:37.569689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a065a-8e85-4cc7-a1d4-bcd599a958d1 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:37.598377 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d5168-5ee6-4f92-9fae-e0222ba8c6f8 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:37.602862 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:37.636925 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905ca48-4ac6-4c6b-bb87-22c19cfbda66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:37.637538 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905ca48-4ac6-4c6b-bb87-22c19cfbda66 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:37.637813 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905ca48-4ac6-4c6b-bb87-22c19cfbda66 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:37.638083 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905ca48-4ac6-4c6b-bb87-22c19cfbda66 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:37.638728 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905ca48-4ac6-4c6b-bb87-22c19cfbda66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:37.639026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905ca48-4ac6-4c6b-bb87-22c19cfbda66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:37.640476 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905ca48-4ac6-4c6b-bb87-22c19cfbda66 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:37.657678 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 488/971] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:37.660510 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905ca48-4ac6-4c6b-bb87-22c19cfbda66 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:37.703146 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a065a-8e85-4cc7-a1d4-bcd599a958d1 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:37.746417 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8826be-4a37-4d4a-a5ce-e57e66875b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:37.747857 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8826be-4a37-4d4a-a5ce-e57e66875b28 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:37.749133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8826be-4a37-4d4a-a5ce-e57e66875b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:37.749808 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8826be-4a37-4d4a-a5ce-e57e66875b28 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:37.750826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8826be-4a37-4d4a-a5ce-e57e66875b28 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:37.753381 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8826be-4a37-4d4a-a5ce-e57e66875b28 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:37.767342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:37.786297 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905ca48-4ac6-4c6b-bb87-22c19cfbda66 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:37.799945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.sql.core [None req-8c8826be-4a37-4d4a-a5ce-e57e66875b28 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 08:24:37.799945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 08 08:24:37.799945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [parameters: {'user_id': 'a2c1ed1091384202b86d10ee6e9b8565', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 08 08:24:37.799945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80262) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 08:24:37.806624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-8c8826be-4a37-4d4a-a5ce-e57e66875b28 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 08 08:24:37.806624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:37.806624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 08:24:37.806624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 08:24:37.806624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 08:24:37.806624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 08:24:37.806624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 08:24:37.806624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 08:24:37.806624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 08:24:37.806624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application conn.query(q) Aug 08 08:24:37.806624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 08:24:37.806624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 08:24:37.806624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 08:24:37.806624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application result.read() Aug 08 08:24:37.806624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 08:24:37.807606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 08:24:37.807606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 08:24:37.807606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 08:24:37.807606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 08:24:37.807606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 08:24:37.807606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 08:24:37.807606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 08:24:37.807606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 08:24:37.807606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:37.807606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 08:24:37.807606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:37.807606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:37.807606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 08:24:37.807606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 08:24:37.807606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 08 08:24:37.807606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 08 08:24:37.807606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 08:24:37.807606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application next(self.gen) Aug 08 08:24:37.808635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 08:24:37.808635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application yield resource Aug 08 08:24:37.808635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 08:24:37.808635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application next(self.gen) Aug 08 08:24:37.808635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 08:24:37.808635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 08:24:37.808635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 08:24:37.808635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 08:24:37.808635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 08:24:37.808635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise self.value Aug 08 08:24:37.808635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 08:24:37.808635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 08:24:37.808635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 08:24:37.808635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application session.commit() Aug 08 08:24:37.808635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 08:24:37.808635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 08:24:37.808635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 08:24:37.810250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 08:24:37.810250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 08:24:37.810250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self.session.flush() Aug 08 08:24:37.810250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 08:24:37.810250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 08:24:37.810250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 08:24:37.810250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 08:24:37.810250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 08:24:37.810250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application compat.raise_( Aug 08 08:24:37.810250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 08:24:37.810250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception Aug 08 08:24:37.810250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 08:24:37.810250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 08:24:37.810250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 08:24:37.810250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 08:24:37.810250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 08:24:37.810250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 08:24:37.812200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 08:24:37.812200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 08:24:37.812200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 08 08:24:37.812200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 08 08:24:37.812200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 08:24:37.812200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 08:24:37.812200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 08:24:37.812200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 08:24:37.812200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 08:24:37.812200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 08:24:37.812200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 08:24:37.812200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 08:24:37.812200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 08:24:37.812200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 08:24:37.812200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 08:24:37.812200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception Aug 08 08:24:37.814493 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 08:24:37.814493 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 08:24:37.814493 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 08:24:37.814493 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 08:24:37.814493 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 08:24:37.814493 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 08:24:37.814493 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 08:24:37.814493 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application conn.query(q) Aug 08 08:24:37.814493 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 08:24:37.814493 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 08:24:37.814493 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 08:24:37.814493 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application result.read() Aug 08 08:24:37.814493 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 08:24:37.814493 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 08:24:37.814493 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 08:24:37.814493 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 08:24:37.816310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 08:24:37.816310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 08:24:37.816310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 08:24:37.816310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 08:24:37.816310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 08:24:37.816310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 08 08:24:37.816310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a2c1ed1091384202b86d10ee6e9b8565', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 08 08:24:37.816310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 08:24:37.816310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:37.816310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:24:37.816310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:37.816310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:37.816310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:37.816310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:37.816310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:37.818240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:37.818240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:37.818240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:37.818240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:37.818240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:37.818240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:37.818240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:37.818240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 08 08:24:37.818240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 08 08:24:37.818240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:37.818240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:37.818240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:24:37.818240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:37.818240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:24:37.818240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:37.818240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 08 08:24:37.818240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 08 08:24:37.821444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 08 08:24:37.821444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 08 08:24:37.821444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 08:24:37.821444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 08:24:37.821444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 08 08:24:37.821444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:37.821444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 484/972] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Aug 8 08:24:37 2022] POST /identity/v3/users => generated 160 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:24:37.821444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7ba31-77de-4499-92b7-632827d9e123 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:37.821444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7ba31-77de-4499-92b7-632827d9e123 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:37.821444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7ba31-77de-4499-92b7-632827d9e123 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:37.822996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 485/973] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:37.822996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e7ba31-77de-4499-92b7-632827d9e123 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:37.837788 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51ea5d-0ba8-4c9c-a540-c95acfd156a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:37.838441 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51ea5d-0ba8-4c9c-a540-c95acfd156a6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:37.838718 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51ea5d-0ba8-4c9c-a540-c95acfd156a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:37.838987 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51ea5d-0ba8-4c9c-a540-c95acfd156a6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:37.850211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51ea5d-0ba8-4c9c-a540-c95acfd156a6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:37.851660 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51ea5d-0ba8-4c9c-a540-c95acfd156a6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:37.863842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-2c51ea5d-0ba8-4c9c-a540-c95acfd156a6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 08:24:37.863842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:37.863842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:37.863842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:37.863842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:37.863842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:37.863842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:37.863842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:37.863842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:37.863842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:37.863842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:37.863842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:37.863842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:24:37.863842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:24:37.863842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:24:37.863842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:24:37.864831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:37.864831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:37.864831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:24:37.864831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:37.864831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:24:37.864831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:37.864831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:37.864831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:37.864831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:37.864831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:37.864831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:37.864831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:37.864831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:37.864831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:37.864831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:37.864831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:37.864831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:37.864831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:37.870270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:37.870270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:37.870270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:24:37.870270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:24:37.870270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:24:37.870270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:24:37.870270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:24:37.870270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:24:37.870270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 08:24:37.870270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:37.870270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 489/974] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Aug 8 08:24:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:24:37.870270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f58777-2b64-4b03-98c7-ef979a295560 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:37.870270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f58777-2b64-4b03-98c7-ef979a295560 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:37.871005 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f58777-2b64-4b03-98c7-ef979a295560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:37.885419 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-78f58777-2b64-4b03-98c7-ef979a295560 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:37.886603 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-78f58777-2b64-4b03-98c7-ef979a295560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:37.927565 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:37.938172 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e7ba31-77de-4499-92b7-632827d9e123 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:37.957924 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 490/975] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:37 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:37.958982 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f58777-2b64-4b03-98c7-ef979a295560 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:37.980959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178bd6c-7bcd-4605-bda6-19b8b5f3a698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:37.981968 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178bd6c-7bcd-4605-bda6-19b8b5f3a698 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:37.982655 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178bd6c-7bcd-4605-bda6-19b8b5f3a698 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:37.983987 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178bd6c-7bcd-4605-bda6-19b8b5f3a698 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:37.994599 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9178bd6c-7bcd-4605-bda6-19b8b5f3a698 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:37.996286 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9178bd6c-7bcd-4605-bda6-19b8b5f3a698 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:38.008037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-9178bd6c-7bcd-4605-bda6-19b8b5f3a698 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 08:24:38.008037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:38.008037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:38.008037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:38.008037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:38.008037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:38.008037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:38.008037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:38.008037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:38.008037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:38.008037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:38.008037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:38.008037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:24:38.008037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:24:38.008037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:24:38.008037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:24:38.009004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:38.009004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:38.009004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:24:38.009004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:38.009004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:24:38.009004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:38.009004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:38.009004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:38.009004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:38.009004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:38.009004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:38.009004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:38.009004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:38.009004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:38.009004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:38.009004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:38.009004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:38.009004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:38.010300 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:38.010300 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:38.010300 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:24:38.010300 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:24:38.010300 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:24:38.010300 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:24:38.010300 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:24:38.010300 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:24:38.010300 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 08:24:38.010300 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:38.013872 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 486/976] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:24:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:38.015382 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178bd6c-7bcd-4605-bda6-19b8b5f3a698 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:38.051894 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:38.081568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f58777-2b64-4b03-98c7-ef979a295560 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:38.118594 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b33a9e2-b87a-4284-bb51-1840001eba78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:38.119202 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33a9e2-b87a-4284-bb51-1840001eba78 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:38.119487 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33a9e2-b87a-4284-bb51-1840001eba78 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:38.119759 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33a9e2-b87a-4284-bb51-1840001eba78 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:38.120483 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b33a9e2-b87a-4284-bb51-1840001eba78 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:38.120788 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b33a9e2-b87a-4284-bb51-1840001eba78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:38.120900 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178bd6c-7bcd-4605-bda6-19b8b5f3a698 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:38.122356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b33a9e2-b87a-4284-bb51-1840001eba78 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:38.146749 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 491/977] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Aug 8 08:24:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:38.147411 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b33a9e2-b87a-4284-bb51-1840001eba78 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:38.156274 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fecc11-b653-4463-b6dc-59d58ead7d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:38.156915 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fecc11-b653-4463-b6dc-59d58ead7d05 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:38.157221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fecc11-b653-4463-b6dc-59d58ead7d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:38.157495 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fecc11-b653-4463-b6dc-59d58ead7d05 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:38.158076 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fecc11-b653-4463-b6dc-59d58ead7d05 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:38.159432 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fecc11-b653-4463-b6dc-59d58ead7d05 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:38.168044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-40fecc11-b653-4463-b6dc-59d58ead7d05 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 08:24:38.168044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:38.168044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 08:24:38.168044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 08:24:38.168044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 08:24:38.168044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 08:24:38.168044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 08:24:38.168044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:38.168044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 08:24:38.168044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:38.168044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:24:38.168044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:38.168044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:38.168044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:38.168044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:38.168044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:38.168044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:38.169061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:38.169061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:38.169061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:38.169061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:38.169061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:38.169061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:38.169061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 08:24:38.169061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 08:24:38.169061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 08:24:38.169061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 08:24:38.169061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:38.169061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:38.169061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:38.169061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:38.169061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:38.169061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:38.169947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:38.169947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:38.169947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:38.169947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:38.169947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:38.169947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:38.169947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:38.169947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:38.169947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:38.169947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:38.169947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 08:24:38.169947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:38.169947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 08:24:38.169947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:38.169947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 487/978] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Aug 8 08:24:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:38.170788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fecc11-b653-4463-b6dc-59d58ead7d05 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:38.194343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:38.261716 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b33a9e2-b87a-4284-bb51-1840001eba78 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:38.282133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fecc11-b653-4463-b6dc-59d58ead7d05 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:38.293683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:38.299525 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d206068-3140-426e-bb77-1003c8de2097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:38.300296 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d206068-3140-426e-bb77-1003c8de2097 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:38.300575 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d206068-3140-426e-bb77-1003c8de2097 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:38.300958 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d206068-3140-426e-bb77-1003c8de2097 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:38.305538 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d206068-3140-426e-bb77-1003c8de2097 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:38.305538 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d206068-3140-426e-bb77-1003c8de2097 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:38.305538 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d206068-3140-426e-bb77-1003c8de2097 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:38.317966 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87e817-ebfe-4300-9337-662e87515d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:38.318576 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87e817-ebfe-4300-9337-662e87515d31 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:38.320694 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87e817-ebfe-4300-9337-662e87515d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:38.320694 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87e817-ebfe-4300-9337-662e87515d31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:38.347824 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 492/979] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 08:24:38 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:38.358991 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d206068-3140-426e-bb77-1003c8de2097 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:38.404230 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af87e817-ebfe-4300-9337-662e87515d31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:38.407007 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af87e817-ebfe-4300-9337-662e87515d31 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:38.430387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:38.505720 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d206068-3140-426e-bb77-1003c8de2097 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:38.527727 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 488/980] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:24:38 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 08:24:38.536121 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87e817-ebfe-4300-9337-662e87515d31 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:38.553184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6bfa69-31b9-40c5-8523-eccd3d1fb7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:38.554052 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bfa69-31b9-40c5-8523-eccd3d1fb7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:38.554466 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bfa69-31b9-40c5-8523-eccd3d1fb7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:38.554892 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bfa69-31b9-40c5-8523-eccd3d1fb7c5 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:38.555713 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6bfa69-31b9-40c5-8523-eccd3d1fb7c5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:38.556181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6bfa69-31b9-40c5-8523-eccd3d1fb7c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:38.557727 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6bfa69-31b9-40c5-8523-eccd3d1fb7c5 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:38.564273 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 493/981] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Aug 8 08:24:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:38.571184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6bfa69-31b9-40c5-8523-eccd3d1fb7c5 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:38.646406 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:38.669984 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87e817-ebfe-4300-9337-662e87515d31 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:38.687907 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:38.698642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6bfa69-31b9-40c5-8523-eccd3d1fb7c5 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:38.705774 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1363228-57c1-4d25-8b0b-323ee3a808a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:38.706426 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1363228-57c1-4d25-8b0b-323ee3a808a0 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:38.706683 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1363228-57c1-4d25-8b0b-323ee3a808a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:38.706948 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1363228-57c1-4d25-8b0b-323ee3a808a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:38.717230 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1363228-57c1-4d25-8b0b-323ee3a808a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:38.719365 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1363228-57c1-4d25-8b0b-323ee3a808a0 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:38.735571 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-c1363228-57c1-4d25-8b0b-323ee3a808a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:38.735571 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:38.735571 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:38.735571 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:38.735571 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:38.735571 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:38.735571 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:38.735571 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:38.735571 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:38.735571 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:38.735571 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:38.735571 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:38.735571 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:38.735571 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:38.735571 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:38.735571 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:38.736574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:38.736574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:38.736574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:38.736574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:38.736574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:38.736574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:38.736574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:38.736574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:38.736574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:38.736574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:38.736574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:38.736574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:38.736574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:38.736574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:38.736574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:38.736574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:38.736574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:38.742418 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:38.742418 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:38.742418 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:38.742418 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 489/982] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:38.742418 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd918f2-4a84-49e5-8d90-ba3af42842cf None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:38.742418 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd918f2-4a84-49e5-8d90-ba3af42842cf None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:38.742418 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd918f2-4a84-49e5-8d90-ba3af42842cf None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:38.742418 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 490/983] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:38.742899 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e306d8-568a-4725-a9b6-c89b3b77cc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:38.742899 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e306d8-568a-4725-a9b6-c89b3b77cc02 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:38.742899 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e306d8-568a-4725-a9b6-c89b3b77cc02 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:38.742899 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e306d8-568a-4725-a9b6-c89b3b77cc02 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:38.743170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e306d8-568a-4725-a9b6-c89b3b77cc02 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:38.743170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e306d8-568a-4725-a9b6-c89b3b77cc02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:38.743292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd918f2-4a84-49e5-8d90-ba3af42842cf None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:38.744134 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e306d8-568a-4725-a9b6-c89b3b77cc02 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:38.755306 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 494/984] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Aug 8 08:24:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:38.755306 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4443d-59c2-4111-ab3c-9af0856f87b6 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:38.755306 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4443d-59c2-4111-ab3c-9af0856f87b6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:38.755306 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4443d-59c2-4111-ab3c-9af0856f87b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:38.770698 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-5fa4443d-59c2-4111-ab3c-9af0856f87b6 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:38.771817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5fa4443d-59c2-4111-ab3c-9af0856f87b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:38.838121 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 495/985] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:38 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:38.839256 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4443d-59c2-4111-ab3c-9af0856f87b6 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:38.845798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd918f2-4a84-49e5-8d90-ba3af42842cf None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:38.880364 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbada38-d626-4ae4-bb9b-24f46bda4b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:38.880972 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbada38-d626-4ae4-bb9b-24f46bda4b12 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:38.881282 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbada38-d626-4ae4-bb9b-24f46bda4b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:38.881556 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbada38-d626-4ae4-bb9b-24f46bda4b12 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:38.882196 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbada38-d626-4ae4-bb9b-24f46bda4b12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:38.882493 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbada38-d626-4ae4-bb9b-24f46bda4b12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:38.883839 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbada38-d626-4ae4-bb9b-24f46bda4b12 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:38.887458 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:38.899953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 491/986] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:38.899953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbada38-d626-4ae4-bb9b-24f46bda4b12 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:38.924562 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:38.937514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4443d-59c2-4111-ab3c-9af0856f87b6 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:38.970593 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a90f00-7668-4311-93b9-f7fd1ba08905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:38.971202 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a90f00-7668-4311-93b9-f7fd1ba08905 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:38.971478 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a90f00-7668-4311-93b9-f7fd1ba08905 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:38.971739 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a90f00-7668-4311-93b9-f7fd1ba08905 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:38.972436 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a90f00-7668-4311-93b9-f7fd1ba08905 None admin] RBAC: Adding query filter params (interface=admin, service_id=1e921f2143df4fa1aa98e2f87ac53434, region_id=RegionOne) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:38.972733 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a90f00-7668-4311-93b9-f7fd1ba08905 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:38.974122 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a90f00-7668-4311-93b9-f7fd1ba08905 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:38.978979 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 496/987] 10.222.0.33 () {56 vars in 1249 bytes} [Mon Aug 8 08:24:38 2022] GET /identity/v3/endpoints?service_id=1e921f2143df4fa1aa98e2f87ac53434&interface=admin®ion_id=RegionOne => generated 193 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:38.979738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a90f00-7668-4311-93b9-f7fd1ba08905 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:38.996976 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbada38-d626-4ae4-bb9b-24f46bda4b12 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:39.031959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248760e-367d-47d8-bf16-b9798629a7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:39.032507 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248760e-367d-47d8-bf16-b9798629a7af None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:39.032786 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248760e-367d-47d8-bf16-b9798629a7af None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:39.033056 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248760e-367d-47d8-bf16-b9798629a7af None admin] PATH_INFO: `/v3/services/compute` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:39.033665 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e248760e-367d-47d8-bf16-b9798629a7af None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:39.035012 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e248760e-367d-47d8-bf16-b9798629a7af None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:39.043375 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-e248760e-367d-47d8-bf16-b9798629a7af None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 08:24:39.043375 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:39.043375 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 08:24:39.043375 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 08:24:39.043375 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 08:24:39.043375 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 08:24:39.043375 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 08:24:39.043375 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:39.043375 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 08:24:39.043375 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:39.043375 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:24:39.043375 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:39.043375 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:39.043375 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:39.043375 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:39.043375 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:39.043375 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:39.044372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:39.044372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:39.044372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:39.044372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:39.044372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:39.044372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:39.044372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 08:24:39.044372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 08:24:39.044372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 08:24:39.044372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 08:24:39.044372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:39.044372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:39.044372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:39.044372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:39.044372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:39.044372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:39.045291 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:39.045291 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:39.045291 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:39.045291 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:39.045291 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:39.045291 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:39.045291 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:39.045291 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:39.045291 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:39.045291 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:39.045291 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 08:24:39.045291 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:39.045291 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 08:24:39.045291 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:39.045291 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 492/988] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:24:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:39.046267 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248760e-367d-47d8-bf16-b9798629a7af None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:39.081223 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a90f00-7668-4311-93b9-f7fd1ba08905 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:39.094716 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:39.115340 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f77aef5-e90c-4b3a-811e-d6d28efdab7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:39.115952 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77aef5-e90c-4b3a-811e-d6d28efdab7a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:39.116228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77aef5-e90c-4b3a-811e-d6d28efdab7a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:39.116498 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77aef5-e90c-4b3a-811e-d6d28efdab7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:39.122972 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f77aef5-e90c-4b3a-811e-d6d28efdab7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:39.124522 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f77aef5-e90c-4b3a-811e-d6d28efdab7a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:39.135849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-8f77aef5-e90c-4b3a-811e-d6d28efdab7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:39.135849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:39.135849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:39.135849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:39.135849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:39.135849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:39.135849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:39.135849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:39.135849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:39.135849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:39.135849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:39.135849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:39.135849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:24:39.135849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:24:39.135849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:24:39.136828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:24:39.136828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:39.136828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:39.136828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:39.136828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:39.136828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:39.136828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:39.136828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:39.136828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:39.136828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:39.136828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:39.136828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:39.136828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:39.136828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:39.136828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:39.136828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:39.136828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:39.137838 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:24:39.137838 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:24:39.137838 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:24:39.137838 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:24:39.137838 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:24:39.137838 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:24:39.137838 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:39.137838 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:39.137838 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 497/989] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:24:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:39.137838 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f77aef5-e90c-4b3a-811e-d6d28efdab7a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:39.138898 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:39.150310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248760e-367d-47d8-bf16-b9798629a7af None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:39.186868 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b1235-b9d2-40c0-a700-3dd1032e3f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:39.187726 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b1235-b9d2-40c0-a700-3dd1032e3f62 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:39.188141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b1235-b9d2-40c0-a700-3dd1032e3f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:39.188476 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b1235-b9d2-40c0-a700-3dd1032e3f62 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:39.189107 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b1235-b9d2-40c0-a700-3dd1032e3f62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:39.190499 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b1235-b9d2-40c0-a700-3dd1032e3f62 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:39.196915 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 493/990] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Aug 8 08:24:39 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:24:39.197599 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b1235-b9d2-40c0-a700-3dd1032e3f62 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:39.244488 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f77aef5-e90c-4b3a-811e-d6d28efdab7a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:39.292084 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be3f2b-716c-436b-b149-eb4661448040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:39.292923 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be3f2b-716c-436b-b149-eb4661448040 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:39.293231 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be3f2b-716c-436b-b149-eb4661448040 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:39.293510 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be3f2b-716c-436b-b149-eb4661448040 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:39.294336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87be3f2b-716c-436b-b149-eb4661448040 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:39.294647 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87be3f2b-716c-436b-b149-eb4661448040 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:39.297173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87be3f2b-716c-436b-b149-eb4661448040 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:39.303712 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 498/991] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Aug 8 08:24:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:39.309929 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be3f2b-716c-436b-b149-eb4661448040 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:39.330042 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b1235-b9d2-40c0-a700-3dd1032e3f62 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:39.349407 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:39.372600 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb45172-57b8-4825-a4f4-16f91b76e1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:39.373418 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb45172-57b8-4825-a4f4-16f91b76e1a2 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:39.374243 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb45172-57b8-4825-a4f4-16f91b76e1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:39.374715 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb45172-57b8-4825-a4f4-16f91b76e1a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:39.375556 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb45172-57b8-4825-a4f4-16f91b76e1a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:39.376197 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb45172-57b8-4825-a4f4-16f91b76e1a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:39.377316 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb45172-57b8-4825-a4f4-16f91b76e1a2 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:39.391483 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 494/992] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:24:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:39.398222 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb45172-57b8-4825-a4f4-16f91b76e1a2 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:39.422287 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:39.422639 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be3f2b-716c-436b-b149-eb4661448040 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:39.459044 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b77039-05ef-4818-8f67-4bd7d22390a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:39.459646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b77039-05ef-4818-8f67-4bd7d22390a6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:39.460203 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b77039-05ef-4818-8f67-4bd7d22390a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:39.460203 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b77039-05ef-4818-8f67-4bd7d22390a6 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:39.460852 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b77039-05ef-4818-8f67-4bd7d22390a6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:39.461167 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b77039-05ef-4818-8f67-4bd7d22390a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:39.462527 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b77039-05ef-4818-8f67-4bd7d22390a6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:39.467396 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 499/993] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Aug 8 08:24:39 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:39.472880 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b77039-05ef-4818-8f67-4bd7d22390a6 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:39.510308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb45172-57b8-4825-a4f4-16f91b76e1a2 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:39.555610 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429d43f-0c4d-472f-a957-e913b0f9d094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:39.556587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429d43f-0c4d-472f-a957-e913b0f9d094 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:39.556790 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429d43f-0c4d-472f-a957-e913b0f9d094 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:39.557122 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429d43f-0c4d-472f-a957-e913b0f9d094 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:39.558103 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0429d43f-0c4d-472f-a957-e913b0f9d094 None admin] RBAC: Adding query filter params (role.id=b4f3adb27a0048ea86dbf52cb1d2ece5, scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4, user.id=7f3422c6f9a041ceba851b71bd15d0df) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:39.558471 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0429d43f-0c4d-472f-a957-e913b0f9d094 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:39.560208 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0429d43f-0c4d-472f-a957-e913b0f9d094 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:39.566306 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 495/994] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:24:39 2022] GET /identity/v3/role_assignments?user.id=7f3422c6f9a041ceba851b71bd15d0df&scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4&role.id=b4f3adb27a0048ea86dbf52cb1d2ece5 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:39.583233 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:39.587671 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b77039-05ef-4818-8f67-4bd7d22390a6 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:39.626261 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b2358-aab7-4518-bb5c-9a6a976b56d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:39.626899 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b2358-aab7-4518-bb5c-9a6a976b56d3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:39.627179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b2358-aab7-4518-bb5c-9a6a976b56d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:39.627453 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b2358-aab7-4518-bb5c-9a6a976b56d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:39.628168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b2358-aab7-4518-bb5c-9a6a976b56d3 None admin] RBAC: Adding query filter params (interface=public, service_id=3fa5301e3716415fa948e189f15ac18a, region_id=RegionOne) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:39.628472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b2358-aab7-4518-bb5c-9a6a976b56d3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:39.629897 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b2358-aab7-4518-bb5c-9a6a976b56d3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:39.635881 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 500/995] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Aug 8 08:24:39 2022] GET /identity/v3/endpoints?service_id=3fa5301e3716415fa948e189f15ac18a&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:39.641517 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429d43f-0c4d-472f-a957-e913b0f9d094 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:39.733031 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:39.750814 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429d43f-0c4d-472f-a957-e913b0f9d094 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:39.787791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c54440-a329-4275-8065-c3d76d357f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:39.788402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c54440-a329-4275-8065-c3d76d357f59 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:39.788682 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c54440-a329-4275-8065-c3d76d357f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:39.788954 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c54440-a329-4275-8065-c3d76d357f59 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:39.789593 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c54440-a329-4275-8065-c3d76d357f59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:39.790952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c54440-a329-4275-8065-c3d76d357f59 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:39.797337 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 496/996] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Aug 8 08:24:39 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:24:41.219207 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3710571-dc47-4b6e-af8c-7551638d0588 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:41.219772 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3710571-dc47-4b6e-af8c-7551638d0588 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:41.219772 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3710571-dc47-4b6e-af8c-7551638d0588 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:41.221137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 501/997] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:41.226540 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bed9cd-17e0-49f9-8897-dda818be497e None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:41.226945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bed9cd-17e0-49f9-8897-dda818be497e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:41.227118 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bed9cd-17e0-49f9-8897-dda818be497e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:41.245359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-93bed9cd-17e0-49f9-8897-dda818be497e None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:41.246842 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-93bed9cd-17e0-49f9-8897-dda818be497e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:41.314808 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 497/998] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:41 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:41.336509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3710571-dc47-4b6e-af8c-7551638d0588 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:41.441856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3710571-dc47-4b6e-af8c-7551638d0588 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:41.475573 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a4378-b63d-4edb-ac04-e699db517be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:41.476122 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a4378-b63d-4edb-ac04-e699db517be7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:41.476395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a4378-b63d-4edb-ac04-e699db517be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:41.476661 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a4378-b63d-4edb-ac04-e699db517be7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:41.477265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a4378-b63d-4edb-ac04-e699db517be7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:41.478630 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a4378-b63d-4edb-ac04-e699db517be7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:41.488164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-f11a4378-b63d-4edb-ac04-e699db517be7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 08:24:41.488164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:41.488164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 08:24:41.488164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 08:24:41.488164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 08:24:41.488164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 08:24:41.488164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 08:24:41.488164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:41.488164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 08:24:41.488164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:41.488164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:24:41.488164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:41.488164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:41.488164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:41.488164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:41.488164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:41.488164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:41.489264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:41.489264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:41.489264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:41.489264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:41.489264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:41.489264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:41.489264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 08:24:41.489264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 08:24:41.489264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 08:24:41.489264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 08:24:41.489264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:41.489264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:41.489264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:41.489264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:41.489264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:41.489264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:41.490175 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:41.490175 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:41.490175 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:41.490175 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:41.490175 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:41.490175 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:41.490175 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:41.490175 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:41.490175 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:41.490175 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:41.490175 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 08:24:41.490175 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:41.490175 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 08:24:41.490175 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:41.491541 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 502/999] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Aug 8 08:24:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:41.495477 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bed9cd-17e0-49f9-8897-dda818be497e None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:41.567861 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:41.593766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bed9cd-17e0-49f9-8897-dda818be497e None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:41.606264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c15f4-d7f7-4476-ace7-952069ecdc85 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:41.606548 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c15f4-d7f7-4476-ace7-952069ecdc85 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:41.606823 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c15f4-d7f7-4476-ace7-952069ecdc85 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:41.609175 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 503/1000] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:41.613600 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96441078-b442-4dd4-8ca1-a3888e0b6690 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:41.613947 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96441078-b442-4dd4-8ca1-a3888e0b6690 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:41.614284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96441078-b442-4dd4-8ca1-a3888e0b6690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:41.632214 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-96441078-b442-4dd4-8ca1-a3888e0b6690 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:41.632412 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683225f0-f1a8-4902-8180-d026cb632f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:41.632910 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683225f0-f1a8-4902-8180-d026cb632f57 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:41.633196 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683225f0-f1a8-4902-8180-d026cb632f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:41.633423 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-96441078-b442-4dd4-8ca1-a3888e0b6690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:41.633529 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683225f0-f1a8-4902-8180-d026cb632f57 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:41.634082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683225f0-f1a8-4902-8180-d026cb632f57 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:41.634381 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683225f0-f1a8-4902-8180-d026cb632f57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:41.635728 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683225f0-f1a8-4902-8180-d026cb632f57 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:41.641118 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 498/1001] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Aug 8 08:24:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:41.646518 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683225f0-f1a8-4902-8180-d026cb632f57 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:41.657419 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:41.702616 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 504/1002] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:41 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:41.724160 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96441078-b442-4dd4-8ca1-a3888e0b6690 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:41.754109 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683225f0-f1a8-4902-8180-d026cb632f57 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:41.790992 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8231f3e-24e6-43c4-831c-266c831216d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:41.791663 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8231f3e-24e6-43c4-831c-266c831216d8 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:41.791938 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8231f3e-24e6-43c4-831c-266c831216d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:41.792216 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8231f3e-24e6-43c4-831c-266c831216d8 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:41.792832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8231f3e-24e6-43c4-831c-266c831216d8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:41.793200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8231f3e-24e6-43c4-831c-266c831216d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:41.794562 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8231f3e-24e6-43c4-831c-266c831216d8 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:41.799864 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 499/1003] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Aug 8 08:24:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:41.801003 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f0d77-bd3a-47d0-b230-06229f8ac3ec None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:41.801310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f0d77-bd3a-47d0-b230-06229f8ac3ec None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:41.801581 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f0d77-bd3a-47d0-b230-06229f8ac3ec None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:41.805185 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 500/1004] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:41.806268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f0d77-bd3a-47d0-b230-06229f8ac3ec None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:41.808829 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:41.829137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96441078-b442-4dd4-8ca1-a3888e0b6690 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:41.863814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb69315-7001-4264-85b4-651c2e04a5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:41.864441 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb69315-7001-4264-85b4-651c2e04a5c4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:41.864717 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb69315-7001-4264-85b4-651c2e04a5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:41.864993 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb69315-7001-4264-85b4-651c2e04a5c4 None admin] PATH_INFO: `/v3/services/image` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:41.865594 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb69315-7001-4264-85b4-651c2e04a5c4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:41.866956 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb69315-7001-4264-85b4-651c2e04a5c4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:41.876120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-0eb69315-7001-4264-85b4-651c2e04a5c4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 08:24:41.876120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:41.876120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 08:24:41.876120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 08:24:41.876120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 08:24:41.876120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 08:24:41.876120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 08:24:41.876120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:41.876120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 08:24:41.876120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:41.876120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:24:41.876120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:41.876120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:41.876120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:41.876120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:41.876120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:41.876120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:41.877311 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:41.877311 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:41.877311 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:41.877311 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:41.877311 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:41.877311 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:41.877311 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 08:24:41.877311 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 08:24:41.877311 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 08:24:41.877311 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 08:24:41.877311 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:41.877311 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:41.877311 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:41.877311 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:41.877311 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:41.877311 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:41.878842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:41.878842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:41.878842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:41.878842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:41.878842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:41.878842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:41.878842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:41.878842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:41.878842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:41.878842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:41.878842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 08:24:41.878842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:41.878842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 08:24:41.878842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:41.878842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 505/1005] 10.222.0.33 () {56 vars in 1105 bytes} [Mon Aug 8 08:24:41 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:24:41.879640 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8884c-419e-4056-8133-86908b8cfa8a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:41.879640 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8884c-419e-4056-8133-86908b8cfa8a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:41.879640 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8884c-419e-4056-8133-86908b8cfa8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:41.897200 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-09e8884c-419e-4056-8133-86908b8cfa8a None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:41.898333 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-09e8884c-419e-4056-8133-86908b8cfa8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:41.912521 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f0d77-bd3a-47d0-b230-06229f8ac3ec None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:41.914310 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:41.948780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941de827-9f3c-43ca-a51f-fea8d484bda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:41.949429 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941de827-9f3c-43ca-a51f-fea8d484bda1 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:41.949706 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941de827-9f3c-43ca-a51f-fea8d484bda1 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:41.949976 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941de827-9f3c-43ca-a51f-fea8d484bda1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:41.950637 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941de827-9f3c-43ca-a51f-fea8d484bda1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:41.952011 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941de827-9f3c-43ca-a51f-fea8d484bda1 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:41.966474 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 506/1006] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:41 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:41.967405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8884c-419e-4056-8133-86908b8cfa8a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:42.006487 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 501/1007] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:24:41 2022] POST /identity/v3/endpoints => generated 325 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:24:42.007238 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941de827-9f3c-43ca-a51f-fea8d484bda1 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:42.078670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:42.089695 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8884c-419e-4056-8133-86908b8cfa8a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:42.130793 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941de827-9f3c-43ca-a51f-fea8d484bda1 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:42.136969 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f845a1df-3fbb-4692-b775-0d301645e6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:42.137621 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845a1df-3fbb-4692-b775-0d301645e6b2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:42.137897 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845a1df-3fbb-4692-b775-0d301645e6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:42.138169 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845a1df-3fbb-4692-b775-0d301645e6b2 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:42.138802 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f845a1df-3fbb-4692-b775-0d301645e6b2 None admin] RBAC: Adding query filter params (name=image) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:42.139102 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f845a1df-3fbb-4692-b775-0d301645e6b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:42.140800 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f845a1df-3fbb-4692-b775-0d301645e6b2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:42.146662 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 507/1008] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:42.152372 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f845a1df-3fbb-4692-b775-0d301645e6b2 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:42.169792 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:42.176444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed34c7f-0721-4b34-a0bf-6a26299842b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:42.182725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed34c7f-0721-4b34-a0bf-6a26299842b7 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:42.182725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed34c7f-0721-4b34-a0bf-6a26299842b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:42.182725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed34c7f-0721-4b34-a0bf-6a26299842b7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:42.182725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed34c7f-0721-4b34-a0bf-6a26299842b7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:42.182725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed34c7f-0721-4b34-a0bf-6a26299842b7 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:42.190112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-4ed34c7f-0721-4b34-a0bf-6a26299842b7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 08:24:42.190112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:42.190112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 08:24:42.190112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 08:24:42.190112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 08:24:42.190112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 08:24:42.190112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 08:24:42.190112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:42.190112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 08:24:42.190112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:42.190112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:24:42.190112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:42.190112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:42.190112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:42.190112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:42.190112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:42.190112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:42.191294 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:42.191294 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:42.191294 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:42.191294 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:42.191294 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:42.191294 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:42.191294 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 08:24:42.191294 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 08:24:42.191294 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 08:24:42.191294 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 08:24:42.191294 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:42.191294 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:42.191294 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:42.191294 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:42.191294 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:42.191294 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:42.192317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:42.192317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:42.192317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:42.192317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:42.192317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:42.192317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:42.192317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:42.192317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:42.192317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:42.192317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:42.192317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 08:24:42.192317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:42.192317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 08:24:42.192317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:42.192317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 502/1009] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:24:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:42.197882 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed34c7f-0721-4b34-a0bf-6a26299842b7 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:42.265220 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f845a1df-3fbb-4692-b775-0d301645e6b2 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:42.299628 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31647e4a-dacf-4765-bb82-6d79521c54eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:42.300232 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31647e4a-dacf-4765-bb82-6d79521c54eb None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:42.300513 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31647e4a-dacf-4765-bb82-6d79521c54eb None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:42.300778 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31647e4a-dacf-4765-bb82-6d79521c54eb None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:42.301425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31647e4a-dacf-4765-bb82-6d79521c54eb None admin] RBAC: Adding query filter params (type=image) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:42.301723 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31647e4a-dacf-4765-bb82-6d79521c54eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:42.303083 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31647e4a-dacf-4765-bb82-6d79521c54eb None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:42.307815 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 508/1010] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:42 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:42.312185 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed34c7f-0721-4b34-a0bf-6a26299842b7 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:42.337556 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:42.351227 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd5b3c-2c22-409e-92af-a52cd39f7914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:42.351934 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd5b3c-2c22-409e-92af-a52cd39f7914 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:42.352181 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd5b3c-2c22-409e-92af-a52cd39f7914 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:42.352451 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd5b3c-2c22-409e-92af-a52cd39f7914 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:42.353091 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bd5b3c-2c22-409e-92af-a52cd39f7914 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:42.353980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bd5b3c-2c22-409e-92af-a52cd39f7914 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:42.354863 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bd5b3c-2c22-409e-92af-a52cd39f7914 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:42.365532 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 503/1011] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Aug 8 08:24:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:42.367561 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31647e4a-dacf-4765-bb82-6d79521c54eb None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:42.457173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:42.481808 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31647e4a-dacf-4765-bb82-6d79521c54eb None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:42.517961 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894e35d-3858-4e78-a3d4-d5e8b1e75ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:42.518488 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894e35d-3858-4e78-a3d4-d5e8b1e75ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:42.518762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894e35d-3858-4e78-a3d4-d5e8b1e75ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:42.519026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894e35d-3858-4e78-a3d4-d5e8b1e75ee7 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:42.519649 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6894e35d-3858-4e78-a3d4-d5e8b1e75ee7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:42.519943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6894e35d-3858-4e78-a3d4-d5e8b1e75ee7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:42.521322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6894e35d-3858-4e78-a3d4-d5e8b1e75ee7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:42.530777 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 509/1012] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Aug 8 08:24:42 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:42.532019 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd5b3c-2c22-409e-92af-a52cd39f7914 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:42.634645 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd5b3c-2c22-409e-92af-a52cd39f7914 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:42.668621 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f4d2b-e605-4b92-83a9-67249e13cb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:42.669260 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f4d2b-e605-4b92-83a9-67249e13cb16 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:42.669541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f4d2b-e605-4b92-83a9-67249e13cb16 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:42.669813 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f4d2b-e605-4b92-83a9-67249e13cb16 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:42.671031 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8f4d2b-e605-4b92-83a9-67249e13cb16 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:42.672394 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8f4d2b-e605-4b92-83a9-67249e13cb16 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:42.760829 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 504/1013] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:24:42 2022] POST /identity/v3/endpoints => generated 334 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:24:44.063229 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29228a-daad-4861-af26-260d8ba348b5 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:44.063789 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29228a-daad-4861-af26-260d8ba348b5 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:44.063789 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29228a-daad-4861-af26-260d8ba348b5 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:44.065302 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 510/1014] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:44.070280 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeca320-688f-4fc3-9f28-b24061020624 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:44.070666 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeca320-688f-4fc3-9f28-b24061020624 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:44.070835 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeca320-688f-4fc3-9f28-b24061020624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:44.089334 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-afeca320-688f-4fc3-9f28-b24061020624 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:44.090515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-afeca320-688f-4fc3-9f28-b24061020624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:44.126123 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:44.164845 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 505/1015] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:44 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:44.187636 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29228a-daad-4861-af26-260d8ba348b5 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:44.294178 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281b630-3f9a-4c78-a3b9-84b04e1310c1 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:44.294375 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29228a-daad-4861-af26-260d8ba348b5 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:44.294459 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281b630-3f9a-4c78-a3b9-84b04e1310c1 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:44.294708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281b630-3f9a-4c78-a3b9-84b04e1310c1 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:44.296703 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 506/1016] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 08:24:44.301921 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88093e92-600a-4532-9674-8cc2a31886e2 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:44.302205 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88093e92-600a-4532-9674-8cc2a31886e2 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:44.302477 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88093e92-600a-4532-9674-8cc2a31886e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:44.320009 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-88093e92-600a-4532-9674-8cc2a31886e2 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:44.321092 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-88093e92-600a-4532-9674-8cc2a31886e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:44.330107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b686c5-1f0f-42f3-891b-b3ed278491de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:44.330750 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b686c5-1f0f-42f3-891b-b3ed278491de None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:44.331033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b686c5-1f0f-42f3-891b-b3ed278491de None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:44.331306 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b686c5-1f0f-42f3-891b-b3ed278491de None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:44.332420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b686c5-1f0f-42f3-891b-b3ed278491de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:44.334143 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b686c5-1f0f-42f3-891b-b3ed278491de None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:44.335713 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 511/1017] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:44.342006 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b686c5-1f0f-42f3-891b-b3ed278491de None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:44.353022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:44.397539 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 507/1018] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:44 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 08:24:44.419815 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88093e92-600a-4532-9674-8cc2a31886e2 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:44.463833 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b686c5-1f0f-42f3-891b-b3ed278491de None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:44.499961 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf26a2a-f5f0-4c6f-b81e-2ace599b11f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:44.500576 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf26a2a-f5f0-4c6f-b81e-2ace599b11f3 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:44.500858 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf26a2a-f5f0-4c6f-b81e-2ace599b11f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:44.501133 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf26a2a-f5f0-4c6f-b81e-2ace599b11f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:44.501808 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf26a2a-f5f0-4c6f-b81e-2ace599b11f3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:44.503236 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf26a2a-f5f0-4c6f-b81e-2ace599b11f3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:44.521700 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:44.525225 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88093e92-600a-4532-9674-8cc2a31886e2 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:44.559884 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd7145-4afe-4a1c-9883-794f761ed3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:44.560489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd7145-4afe-4a1c-9883-794f761ed3d2 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:44.560765 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd7145-4afe-4a1c-9883-794f761ed3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:44.561065 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd7145-4afe-4a1c-9883-794f761ed3d2 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:44.561740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd7145-4afe-4a1c-9883-794f761ed3d2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:44.563090 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd7145-4afe-4a1c-9883-794f761ed3d2 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:44.578330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 512/1019] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Aug 8 08:24:44 2022] POST /identity/v3/projects => generated 311 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:24:44.658995 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 508/1020] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Aug 8 08:24:44 2022] POST /identity/v3/services => generated 242 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:24:44.866454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668793a-f96f-48d7-bd46-2c5169fd633a None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:44.866817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668793a-f96f-48d7-bd46-2c5169fd633a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:44.867010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668793a-f96f-48d7-bd46-2c5169fd633a None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:44.868602 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 513/1021] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:44.873477 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3107c74-1a35-4368-86d7-4a02071703d1 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:44.873914 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3107c74-1a35-4368-86d7-4a02071703d1 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:44.874186 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3107c74-1a35-4368-86d7-4a02071703d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:44.892375 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-f3107c74-1a35-4368-86d7-4a02071703d1 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:44.893576 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f3107c74-1a35-4368-86d7-4a02071703d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:44.973192 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 509/1022] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:44.995908 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668793a-f96f-48d7-bd46-2c5169fd633a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:45.059383 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:45.104447 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668793a-f96f-48d7-bd46-2c5169fd633a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:45.140949 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7097c8-3fbc-42dc-b509-0e3223dd5850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:45.141538 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7097c8-3fbc-42dc-b509-0e3223dd5850 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:45.141961 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7097c8-3fbc-42dc-b509-0e3223dd5850 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:45.142240 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7097c8-3fbc-42dc-b509-0e3223dd5850 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:45.153605 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7097c8-3fbc-42dc-b509-0e3223dd5850 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:24:45.159002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-df7097c8-3fbc-42dc-b509-0e3223dd5850 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 08:24:45.159002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:45.159002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:45.159002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:45.159002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:45.159002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:45.159002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:45.159002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:45.159002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:45.159002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:45.159002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:45.159002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:45.159002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:24:45.159002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:24:45.159002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:24:45.159002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:24:45.159943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:24:45.159943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:24:45.159943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:24:45.159943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:24:45.159943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:45.159943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:45.159943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:45.159943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:45.159943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:45.159943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:45.159943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:45.159943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:45.159943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:45.159943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:45.159943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:45.159943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:45.160807 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:45.160807 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:45.160807 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:45.160807 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:45.160807 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:24:45.160807 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:24:45.160807 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:24:45.160807 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:24:45.160807 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:24:45.160807 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:24:45.160807 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 08:24:45.160807 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:45.165223 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 514/1023] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Aug 8 08:24:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:45.166812 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3107c74-1a35-4368-86d7-4a02071703d1 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:45.253865 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:45.270385 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3107c74-1a35-4368-86d7-4a02071703d1 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:45.305512 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e845a4c-41c7-446c-818a-340b70399b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:45.306266 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e845a4c-41c7-446c-818a-340b70399b4e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:45.306544 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e845a4c-41c7-446c-818a-340b70399b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:45.306814 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e845a4c-41c7-446c-818a-340b70399b4e None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:45.307601 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e845a4c-41c7-446c-818a-340b70399b4e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:45.307910 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e845a4c-41c7-446c-818a-340b70399b4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:45.309284 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e845a4c-41c7-446c-818a-340b70399b4e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:45.319376 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 510/1024] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Aug 8 08:24:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:45.324964 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7097c8-3fbc-42dc-b509-0e3223dd5850 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:45.435613 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7097c8-3fbc-42dc-b509-0e3223dd5850 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:45.456472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:45.470514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17431533-eb87-4776-bf03-f6f57e299213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:45.471150 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17431533-eb87-4776-bf03-f6f57e299213 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:45.471433 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17431533-eb87-4776-bf03-f6f57e299213 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:45.471706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17431533-eb87-4776-bf03-f6f57e299213 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:45.484665 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17431533-eb87-4776-bf03-f6f57e299213 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:45.486135 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17431533-eb87-4776-bf03-f6f57e299213 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:45.500620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-17431533-eb87-4776-bf03-f6f57e299213 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:45.500620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:45.500620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:45.500620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:45.500620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:45.500620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:45.500620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:45.500620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:45.500620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:45.500620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:45.500620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:45.500620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:45.500620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:45.500620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:45.500620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:45.500620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:45.506479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:45.506479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:45.506479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:45.506479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:45.506479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:45.506479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:45.506479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:45.506479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:45.506479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:45.506479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:45.506479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:45.506479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:45.506479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:45.506479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:45.506479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:45.506479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:45.506479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:45.507390 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:45.507390 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:45.507390 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:45.507390 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 515/1025] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:24:45.509128 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e845a4c-41c7-446c-818a-340b70399b4e None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:45.610130 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e845a4c-41c7-446c-818a-340b70399b4e None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:45.643203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208773a-2101-4bba-aad0-e0615281bba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:45.643803 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208773a-2101-4bba-aad0-e0615281bba5 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:45.644077 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208773a-2101-4bba-aad0-e0615281bba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:45.644344 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208773a-2101-4bba-aad0-e0615281bba5 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:45.644977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208773a-2101-4bba-aad0-e0615281bba5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:45.645305 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208773a-2101-4bba-aad0-e0615281bba5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:45.646639 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208773a-2101-4bba-aad0-e0615281bba5 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:45.658486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 511/1026] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:45.663954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17431533-eb87-4776-bf03-f6f57e299213 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:45.770312 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17431533-eb87-4776-bf03-f6f57e299213 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:45.806329 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4940d7e3-d990-485c-8ba0-975d60175c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:45.806967 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940d7e3-d990-485c-8ba0-975d60175c3e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:45.807243 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940d7e3-d990-485c-8ba0-975d60175c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:45.807514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940d7e3-d990-485c-8ba0-975d60175c3e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:45.815839 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4940d7e3-d990-485c-8ba0-975d60175c3e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:45.817314 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4940d7e3-d990-485c-8ba0-975d60175c3e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:45.828526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-4940d7e3-d990-485c-8ba0-975d60175c3e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 08:24:45.828526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:45.828526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:45.828526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:45.828526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:45.828526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:45.828526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:45.828526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:45.828526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:45.828526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:45.828526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:45.828526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:45.828526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:24:45.828526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:24:45.828526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:24:45.828526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:24:45.834644 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:45.834644 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:45.834644 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:24:45.834644 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:45.834644 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:24:45.834644 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:45.834644 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:45.834644 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:45.834644 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:45.834644 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:45.834644 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:45.834644 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:45.834644 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:45.834644 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:45.834644 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:45.834644 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:45.834644 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:45.834644 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:45.835866 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:45.835866 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:45.835866 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:24:45.835866 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:24:45.835866 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:24:45.835866 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:24:45.835866 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:24:45.835866 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:24:45.835866 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 08:24:45.835866 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:45.835866 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 516/1027] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Aug 8 08:24:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:45.837976 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208773a-2101-4bba-aad0-e0615281bba5 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:45.848681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:45.941541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208773a-2101-4bba-aad0-e0615281bba5 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:45.976100 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c69ae-0989-4096-a3df-7a86895388d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:45.976940 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c69ae-0989-4096-a3df-7a86895388d6 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:45.977423 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c69ae-0989-4096-a3df-7a86895388d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:45.977874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c69ae-0989-4096-a3df-7a86895388d6 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:45.978886 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c69ae-0989-4096-a3df-7a86895388d6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:45.979393 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c69ae-0989-4096-a3df-7a86895388d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:45.981018 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c69ae-0989-4096-a3df-7a86895388d6 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:45.998150 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 512/1028] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Aug 8 08:24:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:46.009776 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4940d7e3-d990-485c-8ba0-975d60175c3e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:46.042023 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:46.111919 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4940d7e3-d990-485c-8ba0-975d60175c3e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:46.146170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debddac7-0b94-4f23-ab40-d42189152ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:46.146783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debddac7-0b94-4f23-ab40-d42189152ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:46.147061 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debddac7-0b94-4f23-ab40-d42189152ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:46.147329 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debddac7-0b94-4f23-ab40-d42189152ee4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:46.216294 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debddac7-0b94-4f23-ab40-d42189152ee4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:46.218038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debddac7-0b94-4f23-ab40-d42189152ee4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:46.248343 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:46.321853 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 517/1029] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:24:46 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 08:24:46.328812 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c69ae-0989-4096-a3df-7a86895388d6 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:46.406475 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:46.457641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c69ae-0989-4096-a3df-7a86895388d6 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:46.499823 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105c199-02ad-4c57-a8ed-f20ba95d2ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:46.500815 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105c199-02ad-4c57-a8ed-f20ba95d2ada None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:46.501346 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105c199-02ad-4c57-a8ed-f20ba95d2ada None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:46.501810 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105c199-02ad-4c57-a8ed-f20ba95d2ada None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:46.513805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105c199-02ad-4c57-a8ed-f20ba95d2ada None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:46.515284 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105c199-02ad-4c57-a8ed-f20ba95d2ada None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:46.531732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-e105c199-02ad-4c57-a8ed-f20ba95d2ada None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:46.531732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:46.531732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:46.531732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:46.531732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:46.531732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:46.531732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:46.531732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:46.531732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:46.531732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:46.531732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:46.531732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:46.531732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:46.531732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:46.531732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:46.531732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:46.532764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:46.532764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:46.532764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:46.532764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:46.532764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:46.532764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:46.532764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:46.532764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:46.532764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:46.532764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:46.532764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:46.532764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:46.532764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:46.532764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:46.532764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:46.532764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:46.532764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:46.533789 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:46.533789 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:46.533789 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:46.537499 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 513/1030] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:46.539127 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debddac7-0b94-4f23-ab40-d42189152ee4 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:46.627025 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ce773-9139-4f19-abcb-7295349e93e3 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:46.627404 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ce773-9139-4f19-abcb-7295349e93e3 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:46.627604 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ce773-9139-4f19-abcb-7295349e93e3 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:46.629262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 514/1031] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 08:24:46.634200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40656d14-8690-49e1-a753-cbb8ce6d338b None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:46.634497 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40656d14-8690-49e1-a753-cbb8ce6d338b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:46.634747 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40656d14-8690-49e1-a753-cbb8ce6d338b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:46.653977 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debddac7-0b94-4f23-ab40-d42189152ee4 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:46.658107 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-40656d14-8690-49e1-a753-cbb8ce6d338b None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:46.659792 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-40656d14-8690-49e1-a753-cbb8ce6d338b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:46.686121 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:46.692019 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434a4ce1-1944-4898-b62e-8207fd2a4ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:46.692628 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a4ce1-1944-4898-b62e-8207fd2a4ccc None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:46.692900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a4ce1-1944-4898-b62e-8207fd2a4ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:46.693193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a4ce1-1944-4898-b62e-8207fd2a4ccc None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:46.693834 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434a4ce1-1944-4898-b62e-8207fd2a4ccc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:46.694131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434a4ce1-1944-4898-b62e-8207fd2a4ccc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:46.695488 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434a4ce1-1944-4898-b62e-8207fd2a4ccc None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:46.706833 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 518/1032] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:46.709369 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a465a9b-0bff-4861-886a-f94b2461d2b4 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:46.709369 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a465a9b-0bff-4861-886a-f94b2461d2b4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:46.709369 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a465a9b-0bff-4861-886a-f94b2461d2b4 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:46.710197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 519/1033] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:46.712684 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a465a9b-0bff-4861-886a-f94b2461d2b4 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:46.744386 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 515/1034] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:46 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 08:24:46.745813 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6ca03-660d-4baf-8abb-cff5308be208 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:46.746104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6ca03-660d-4baf-8abb-cff5308be208 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:46.746379 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6ca03-660d-4baf-8abb-cff5308be208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:46.769786 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-7ee6ca03-660d-4baf-8abb-cff5308be208 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:46.771011 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-7ee6ca03-660d-4baf-8abb-cff5308be208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:46.818010 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:46.825779 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a465a9b-0bff-4861-886a-f94b2461d2b4 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:46.847804 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 516/1035] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:46 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:46.849111 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee6ca03-660d-4baf-8abb-cff5308be208 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:46.863319 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66b38b-9b48-435d-a016-fa6eb1a45bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:46.864278 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66b38b-9b48-435d-a016-fa6eb1a45bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:46.864572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66b38b-9b48-435d-a016-fa6eb1a45bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:46.864847 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66b38b-9b48-435d-a016-fa6eb1a45bd7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:46.872603 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66b38b-9b48-435d-a016-fa6eb1a45bd7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:46.874217 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66b38b-9b48-435d-a016-fa6eb1a45bd7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:46.886622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-1d66b38b-9b48-435d-a016-fa6eb1a45bd7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:46.886622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:46.886622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:46.886622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:46.886622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:46.886622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:46.886622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:46.886622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:46.886622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:46.886622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:46.886622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:46.886622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:46.886622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:24:46.886622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:24:46.886622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:24:46.887576 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:24:46.887576 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:46.887576 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:46.887576 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:46.887576 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:46.887576 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:46.887576 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:46.887576 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:46.887576 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:46.887576 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:46.887576 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:46.887576 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:46.887576 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:46.887576 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:46.887576 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:46.887576 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:46.887576 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:46.888491 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:24:46.888491 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:24:46.888491 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:24:46.888491 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:24:46.888491 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:24:46.888491 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:24:46.888491 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:46.888491 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:46.888491 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 520/1036] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:24:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:24:46.889007 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66b38b-9b48-435d-a016-fa6eb1a45bd7 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:46.944164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:46.972252 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee6ca03-660d-4baf-8abb-cff5308be208 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:46.997625 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66b38b-9b48-435d-a016-fa6eb1a45bd7 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:47.006499 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d004f9-5e1f-4176-8b2e-f0b6aa32918d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:47.007129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d004f9-5e1f-4176-8b2e-f0b6aa32918d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:47.007406 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d004f9-5e1f-4176-8b2e-f0b6aa32918d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:47.007978 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d004f9-5e1f-4176-8b2e-f0b6aa32918d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:47.008766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d004f9-5e1f-4176-8b2e-f0b6aa32918d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:47.010219 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d004f9-5e1f-4176-8b2e-f0b6aa32918d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:47.011851 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 517/1037] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:47.012610 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d004f9-5e1f-4176-8b2e-f0b6aa32918d None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:47.037985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef724aa-edc7-4138-89b4-0502061f70a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:47.039025 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef724aa-edc7-4138-89b4-0502061f70a7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:47.039514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef724aa-edc7-4138-89b4-0502061f70a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:47.039989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef724aa-edc7-4138-89b4-0502061f70a7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:47.040745 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef724aa-edc7-4138-89b4-0502061f70a7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:47.042171 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef724aa-edc7-4138-89b4-0502061f70a7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:47.052007 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-aef724aa-edc7-4138-89b4-0502061f70a7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 08:24:47.052007 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:47.052007 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 08:24:47.052007 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 08:24:47.052007 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 08:24:47.052007 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 08:24:47.052007 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 08:24:47.052007 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:47.052007 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 08:24:47.052007 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:47.052007 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:24:47.052007 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:47.052007 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:47.052007 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:47.052007 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:47.052007 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:47.052007 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:47.053119 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:47.053119 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:47.053119 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:47.053119 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:47.053119 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:47.053119 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:47.053119 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 08:24:47.053119 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 08:24:47.053119 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 08:24:47.053119 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 08:24:47.053119 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:47.053119 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:47.053119 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:47.053119 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:47.053119 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:47.053119 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:47.059226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:47.059226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:47.059226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:47.059226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:47.059226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:47.059226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:47.059226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:47.059226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:47.059226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:47.059226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:47.059226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 08:24:47.059226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:47.059226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 08:24:47.059226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:47.059226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 521/1038] 10.222.0.33 () {56 vars in 1105 bytes} [Mon Aug 8 08:24:46 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:47.060040 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef724aa-edc7-4138-89b4-0502061f70a7 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:47.067250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:47.123931 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d004f9-5e1f-4176-8b2e-f0b6aa32918d None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:47.158711 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe5c52-bf22-4c67-8280-eb5c7922b173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:47.159243 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe5c52-bf22-4c67-8280-eb5c7922b173 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:47.159519 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe5c52-bf22-4c67-8280-eb5c7922b173 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:47.159785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe5c52-bf22-4c67-8280-eb5c7922b173 None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:47.160461 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe5c52-bf22-4c67-8280-eb5c7922b173 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:47.160759 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe5c52-bf22-4c67-8280-eb5c7922b173 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:47.162203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe5c52-bf22-4c67-8280-eb5c7922b173 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:47.164116 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef724aa-edc7-4138-89b4-0502061f70a7 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:47.173916 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 518/1039] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:24:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:47.174785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe5c52-bf22-4c67-8280-eb5c7922b173 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:47.185135 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:47.203171 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa8562-30c6-46b0-af02-5d31f8ffcec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:47.203781 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa8562-30c6-46b0-af02-5d31f8ffcec6 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:47.204057 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa8562-30c6-46b0-af02-5d31f8ffcec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:47.204324 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa8562-30c6-46b0-af02-5d31f8ffcec6 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:47.204970 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aa8562-30c6-46b0-af02-5d31f8ffcec6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:47.206691 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aa8562-30c6-46b0-af02-5d31f8ffcec6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:47.274022 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe5c52-bf22-4c67-8280-eb5c7922b173 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:47.276293 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:47.285860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 522/1040] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 08:24:47 2022] POST /identity/v3/users => generated 285 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:24:47.286758 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa8562-30c6-46b0-af02-5d31f8ffcec6 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:47.310436 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4ecc1-585c-4435-9eaf-fb61fab4ff23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:47.311905 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4ecc1-585c-4435-9eaf-fb61fab4ff23 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:47.312133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4ecc1-585c-4435-9eaf-fb61fab4ff23 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:47.312628 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4ecc1-585c-4435-9eaf-fb61fab4ff23 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:47.313718 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4ecc1-585c-4435-9eaf-fb61fab4ff23 None admin] RBAC: Adding query filter params (name=image) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:47.314151 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4ecc1-585c-4435-9eaf-fb61fab4ff23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:47.315613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4ecc1-585c-4435-9eaf-fb61fab4ff23 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:47.320979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 519/1041] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:47.327560 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4ecc1-585c-4435-9eaf-fb61fab4ff23 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:47.423002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa8562-30c6-46b0-af02-5d31f8ffcec6 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:47.447109 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4ecc1-585c-4435-9eaf-fb61fab4ff23 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:47.474363 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57fb93-a04d-4844-bc35-7da1306f0e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:47.475322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57fb93-a04d-4844-bc35-7da1306f0e4c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:47.475693 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57fb93-a04d-4844-bc35-7da1306f0e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:47.476136 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57fb93-a04d-4844-bc35-7da1306f0e4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:47.477408 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57fb93-a04d-4844-bc35-7da1306f0e4c None admin] RBAC: Adding query filter params (role.id=b4f3adb27a0048ea86dbf52cb1d2ece5, scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4, user.id=a3317f751ec74bde95ec262394001292) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:47.477938 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57fb93-a04d-4844-bc35-7da1306f0e4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:47.480413 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57fb93-a04d-4844-bc35-7da1306f0e4c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:47.488283 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 523/1042] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:24:47 2022] GET /identity/v3/role_assignments?user.id=a3317f751ec74bde95ec262394001292&scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4&role.id=b4f3adb27a0048ea86dbf52cb1d2ece5 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:47.496235 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed95e13-cd15-4941-8dd3-c8a67755c1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:47.496861 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed95e13-cd15-4941-8dd3-c8a67755c1d7 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:47.497141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed95e13-cd15-4941-8dd3-c8a67755c1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:47.497448 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed95e13-cd15-4941-8dd3-c8a67755c1d7 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:47.498103 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed95e13-cd15-4941-8dd3-c8a67755c1d7 None admin] RBAC: Adding query filter params (type=image) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:47.499793 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed95e13-cd15-4941-8dd3-c8a67755c1d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:47.500725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed95e13-cd15-4941-8dd3-c8a67755c1d7 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:47.507752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 520/1043] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:47 2022] GET /identity/v3/services?type=image => generated 350 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:47.515258 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57fb93-a04d-4844-bc35-7da1306f0e4c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:47.558114 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:47.629660 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57fb93-a04d-4844-bc35-7da1306f0e4c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:47.665979 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50d8c8-ab00-4da6-814d-5cd1625e4dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:47.667018 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50d8c8-ab00-4da6-814d-5cd1625e4dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:47.667504 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50d8c8-ab00-4da6-814d-5cd1625e4dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:47.668312 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50d8c8-ab00-4da6-814d-5cd1625e4dc1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:47.669247 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a50d8c8-ab00-4da6-814d-5cd1625e4dc1 None admin] RBAC: Adding query filter params (interface=public, service_id=cbe5d39fd0654c08a3eeaccb8e351f42, region_id=RegionOne) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:47.669772 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a50d8c8-ab00-4da6-814d-5cd1625e4dc1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:47.671875 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a50d8c8-ab00-4da6-814d-5cd1625e4dc1 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:47.682060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 524/1044] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Aug 8 08:24:47 2022] GET /identity/v3/endpoints?service_id=cbe5d39fd0654c08a3eeaccb8e351f42&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:47.683716 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed95e13-cd15-4941-8dd3-c8a67755c1d7 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:47.718655 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:47.792349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed95e13-cd15-4941-8dd3-c8a67755c1d7 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:47.827321 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a33c2-e3f3-40e2-a360-5cffc715614a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:47.827927 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a33c2-e3f3-40e2-a360-5cffc715614a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:47.828200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a33c2-e3f3-40e2-a360-5cffc715614a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:47.828465 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a33c2-e3f3-40e2-a360-5cffc715614a None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:47.829064 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a33c2-e3f3-40e2-a360-5cffc715614a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:47.830464 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a33c2-e3f3-40e2-a360-5cffc715614a None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:47.837645 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 521/1045] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Aug 8 08:24:47 2022] GET /identity/v3/services => generated 1957 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:24:49.401211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91407ea3-9c3a-4013-835f-f638f8c38f2f None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:49.401733 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91407ea3-9c3a-4013-835f-f638f8c38f2f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:49.401733 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91407ea3-9c3a-4013-835f-f638f8c38f2f None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:49.403118 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 525/1046] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:49.411876 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e53a9-67ea-4812-bfa3-bd0fe835809c None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:49.412767 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e53a9-67ea-4812-bfa3-bd0fe835809c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:49.413276 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e53a9-67ea-4812-bfa3-bd0fe835809c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:49.433738 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-d33e53a9-67ea-4812-bfa3-bd0fe835809c None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:49.434926 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d33e53a9-67ea-4812-bfa3-bd0fe835809c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:49.472819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93d566-7ff9-4e8a-8632-9311883bbc71 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:49.473179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93d566-7ff9-4e8a-8632-9311883bbc71 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:49.473395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93d566-7ff9-4e8a-8632-9311883bbc71 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:49.474857 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 526/1047] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 08:24:49.479466 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca68c5-cb72-4e3b-b410-f821b3232d45 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:49.479746 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca68c5-cb72-4e3b-b410-f821b3232d45 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:49.480019 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca68c5-cb72-4e3b-b410-f821b3232d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:49.498365 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-cdca68c5-cb72-4e3b-b410-f821b3232d45 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:49.499675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-cdca68c5-cb72-4e3b-b410-f821b3232d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:49.505556 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 522/1048] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:49.527679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e53a9-67ea-4812-bfa3-bd0fe835809c None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:49.555505 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:49.570077 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 527/1049] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 08:24:49.592025 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca68c5-cb72-4e3b-b410-f821b3232d45 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:49.622126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:49.629489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e53a9-67ea-4812-bfa3-bd0fe835809c None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:49.664738 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe475f8-d203-4624-aad4-dc8388fedeef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:49.665323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe475f8-d203-4624-aad4-dc8388fedeef None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:49.665603 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe475f8-d203-4624-aad4-dc8388fedeef None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:49.665874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe475f8-d203-4624-aad4-dc8388fedeef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:49.677305 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe475f8-d203-4624-aad4-dc8388fedeef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:24:49.681945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-9fe475f8-d203-4624-aad4-dc8388fedeef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:24:49.681945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:49.681945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:49.681945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:49.681945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:49.681945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:49.681945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:49.681945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:49.681945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:49.681945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:49.681945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:49.681945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:49.681945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:24:49.681945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:24:49.681945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:24:49.681945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:24:49.682883 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:24:49.682883 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:24:49.682883 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:24:49.682883 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:24:49.682883 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:49.682883 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:49.682883 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:49.682883 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:49.682883 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:49.682883 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:49.682883 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:49.682883 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:49.682883 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:49.682883 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:49.682883 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:49.682883 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:49.683750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:49.683750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:49.683750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:49.683750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:49.683750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:24:49.683750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:24:49.683750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:24:49.683750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:24:49.683750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:24:49.683750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:24:49.683750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:24:49.683750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:49.683750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 523/1050] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:24:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:24:49.689444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe475f8-d203-4624-aad4-dc8388fedeef None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:49.703601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca68c5-cb72-4e3b-b410-f821b3232d45 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:49.739388 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009fdf3-24ab-4470-852c-b92f74c52ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:49.740015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009fdf3-24ab-4470-852c-b92f74c52ecc None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:49.740290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009fdf3-24ab-4470-852c-b92f74c52ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:49.740560 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009fdf3-24ab-4470-852c-b92f74c52ecc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:49.752516 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3009fdf3-24ab-4470-852c-b92f74c52ecc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:24:49.757443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-3009fdf3-24ab-4470-852c-b92f74c52ecc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 08:24:49.757443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:49.757443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:49.757443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:49.757443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:49.757443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:49.757443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:49.757443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:49.757443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:49.757443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:49.757443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:49.757443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:49.757443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:24:49.757443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:24:49.757443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:24:49.757443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:24:49.758379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:24:49.758379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:24:49.758379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:24:49.758379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:24:49.758379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:49.758379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:49.758379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:49.758379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:49.758379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:49.758379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:49.758379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:49.758379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:49.758379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:49.758379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:49.758379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:49.758379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:49.759260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:49.759260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:49.759260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:49.759260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:49.759260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:24:49.759260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:24:49.759260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:24:49.759260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:24:49.759260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:24:49.759260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:24:49.759260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 08:24:49.759260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:49.759260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 528/1051] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Aug 8 08:24:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:24:49.764430 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009fdf3-24ab-4470-852c-b92f74c52ecc None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:49.785371 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:49.800856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe475f8-d203-4624-aad4-dc8388fedeef None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:49.841879 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3d9f8-4b92-4f42-8273-b51b76a0fd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:49.843169 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3d9f8-4b92-4f42-8273-b51b76a0fd33 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:49.843464 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3d9f8-4b92-4f42-8273-b51b76a0fd33 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:49.843733 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3d9f8-4b92-4f42-8273-b51b76a0fd33 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:49.844405 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae3d9f8-4b92-4f42-8273-b51b76a0fd33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:49.844711 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae3d9f8-4b92-4f42-8273-b51b76a0fd33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:49.846539 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae3d9f8-4b92-4f42-8273-b51b76a0fd33 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:49.860391 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 524/1052] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:24:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:49.861423 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3d9f8-4b92-4f42-8273-b51b76a0fd33 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:49.867454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:49.878721 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009fdf3-24ab-4470-852c-b92f74c52ecc None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:49.914812 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a2df2-0b78-4fc3-93bb-995aa6c0f531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:49.915351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a2df2-0b78-4fc3-93bb-995aa6c0f531 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:49.915622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a2df2-0b78-4fc3-93bb-995aa6c0f531 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:49.915893 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a2df2-0b78-4fc3-93bb-995aa6c0f531 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:49.916555 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a2df2-0b78-4fc3-93bb-995aa6c0f531 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:49.916849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a2df2-0b78-4fc3-93bb-995aa6c0f531 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:49.918250 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a2df2-0b78-4fc3-93bb-995aa6c0f531 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:49.925983 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 529/1053] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Aug 8 08:24:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:49.927143 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9101ef-022c-442c-adbc-d31c98da8c56 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:49.927422 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9101ef-022c-442c-adbc-d31c98da8c56 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:49.927690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9101ef-022c-442c-adbc-d31c98da8c56 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:49.929076 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 530/1054] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:49.933556 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9101ef-022c-442c-adbc-d31c98da8c56 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:49.963607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3d9f8-4b92-4f42-8273-b51b76a0fd33 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:49.997055 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1206ca3-9aec-4dce-95d9-ac2e8b606fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:49.997709 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1206ca3-9aec-4dce-95d9-ac2e8b606fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:49.997990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1206ca3-9aec-4dce-95d9-ac2e8b606fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:49.998355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1206ca3-9aec-4dce-95d9-ac2e8b606fb2 None admin] PATH_INFO: `/v3/users/11065c949b954f88becd91e63cb5a81e` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:50.011938 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1206ca3-9aec-4dce-95d9-ac2e8b606fb2 None admin] RBAC: Authorizing `identity:get_user(user_id=11065c949b954f88becd91e63cb5a81e)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:50.013530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1206ca3-9aec-4dce-95d9-ac2e8b606fb2 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:50.015927 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 525/1055] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:24:49 2022] GET /identity/v3/users/11065c949b954f88becd91e63cb5a81e => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:50.017138 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c71eb9-d2fc-4d03-a1f9-c4a73edf95a0 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:50.017442 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c71eb9-d2fc-4d03-a1f9-c4a73edf95a0 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:50.017711 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c71eb9-d2fc-4d03-a1f9-c4a73edf95a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:50.029749 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:50.036309 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-35c71eb9-d2fc-4d03-a1f9-c4a73edf95a0 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:50.037583 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-35c71eb9-d2fc-4d03-a1f9-c4a73edf95a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:50.038762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9101ef-022c-442c-adbc-d31c98da8c56 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:50.072136 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73852558-f05d-407c-9506-4efe248dc4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:50.072781 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73852558-f05d-407c-9506-4efe248dc4e3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:50.073066 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73852558-f05d-407c-9506-4efe248dc4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:50.073365 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73852558-f05d-407c-9506-4efe248dc4e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:50.084817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73852558-f05d-407c-9506-4efe248dc4e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:50.086271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73852558-f05d-407c-9506-4efe248dc4e3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:50.101176 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-73852558-f05d-407c-9506-4efe248dc4e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:50.101176 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:50.101176 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:50.101176 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:50.101176 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:50.101176 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:50.101176 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:50.101176 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:50.101176 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:50.101176 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:50.101176 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:50.101176 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:50.101176 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:50.101176 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:50.101176 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:50.101176 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:50.102283 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:50.102283 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:50.102283 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:50.102283 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:50.102283 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:50.102283 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:50.102283 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:50.102283 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:50.102283 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:50.102283 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:50.102283 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:50.102283 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:50.102283 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:50.102283 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:50.102283 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:50.102283 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:50.102283 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:50.103287 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:50.103287 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:50.103287 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:50.104357 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 526/1056] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:50 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:50.105228 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c71eb9-d2fc-4d03-a1f9-c4a73edf95a0 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:50.107160 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 531/1057] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:50.108031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73852558-f05d-407c-9506-4efe248dc4e3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:50.112343 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:50.206875 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c71eb9-d2fc-4d03-a1f9-c4a73edf95a0 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:50.207324 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73852558-f05d-407c-9506-4efe248dc4e3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:50.239805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa39b6-3b9a-4978-ad7c-724ee68fd9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:50.240402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa39b6-3b9a-4978-ad7c-724ee68fd9ac None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:50.240676 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa39b6-3b9a-4978-ad7c-724ee68fd9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:50.240944 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa39b6-3b9a-4978-ad7c-724ee68fd9ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:50.241212 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764d284-9376-4085-b8f1-e157e9d35a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:50.241815 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764d284-9376-4085-b8f1-e157e9d35a1e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:50.242093 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764d284-9376-4085-b8f1-e157e9d35a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:50.242392 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764d284-9376-4085-b8f1-e157e9d35a1e None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:50.243036 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764d284-9376-4085-b8f1-e157e9d35a1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:50.243336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764d284-9376-4085-b8f1-e157e9d35a1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:50.244219 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:50.244699 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764d284-9376-4085-b8f1-e157e9d35a1e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:50.256877 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 532/1058] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:50.257715 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764d284-9376-4085-b8f1-e157e9d35a1e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:50.307573 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa39b6-3b9a-4978-ad7c-724ee68fd9ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:50.309013 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa39b6-3b9a-4978-ad7c-724ee68fd9ac None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:50.319022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:50.356007 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764d284-9376-4085-b8f1-e157e9d35a1e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:50.390433 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637d693-cb09-499a-9fed-4707192a72ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:50.391058 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637d693-cb09-499a-9fed-4707192a72ca None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:50.391333 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637d693-cb09-499a-9fed-4707192a72ca None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:50.391596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637d693-cb09-499a-9fed-4707192a72ca None admin] PATH_INFO: `/v3/services/image` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:50.392168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1637d693-cb09-499a-9fed-4707192a72ca None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:50.393556 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1637d693-cb09-499a-9fed-4707192a72ca None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:50.402011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-1637d693-cb09-499a-9fed-4707192a72ca None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 08:24:50.402011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:50.402011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 08:24:50.402011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 08:24:50.402011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 08:24:50.402011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 08:24:50.402011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 08:24:50.402011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:50.402011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 08:24:50.402011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:50.402011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:24:50.402011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:50.402011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:50.402011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:50.402011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:50.402011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:50.402011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:50.403407 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:50.403407 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:50.403407 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:50.403407 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:50.403407 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:50.403407 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:50.403407 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 08:24:50.403407 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 08:24:50.403407 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 08:24:50.403407 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 08:24:50.403407 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:50.403407 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:50.403407 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:50.403407 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:50.403407 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:50.403407 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:50.404412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:50.404412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:50.404412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:50.404412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:50.404412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:50.404412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:50.404412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:50.404412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:50.404412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:50.404412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:50.404412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 08:24:50.404412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:24:50.404412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 08:24:50.404412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:50.404412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 533/1059] 10.222.0.33 () {56 vars in 1105 bytes} [Mon Aug 8 08:24:50 2022] GET /identity/v3/services/image => generated 86 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:50.405760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637d693-cb09-499a-9fed-4707192a72ca None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:50.405836 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 527/1060] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:24:50 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 08:24:50.414960 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa39b6-3b9a-4978-ad7c-724ee68fd9ac None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:50.446712 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:50.508720 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637d693-cb09-499a-9fed-4707192a72ca None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:50.509449 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa39b6-3b9a-4978-ad7c-724ee68fd9ac None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:50.534096 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:50.541728 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec39a99-92cd-4528-9d11-98d5bc7d7931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:50.542362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec39a99-92cd-4528-9d11-98d5bc7d7931 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:50.542620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec39a99-92cd-4528-9d11-98d5bc7d7931 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:50.542891 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec39a99-92cd-4528-9d11-98d5bc7d7931 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:50.544687 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92304de4-3719-4692-95c7-80082053c0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:50.545310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92304de4-3719-4692-95c7-80082053c0b3 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:50.546277 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92304de4-3719-4692-95c7-80082053c0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:50.546523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92304de4-3719-4692-95c7-80082053c0b3 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:50.547134 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92304de4-3719-4692-95c7-80082053c0b3 None admin] RBAC: Adding query filter params (name=image) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:50.547431 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92304de4-3719-4692-95c7-80082053c0b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:50.548783 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92304de4-3719-4692-95c7-80082053c0b3 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:50.550973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec39a99-92cd-4528-9d11-98d5bc7d7931 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:50.552391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec39a99-92cd-4528-9d11-98d5bc7d7931 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:50.553757 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 528/1061] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:50.554521 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92304de4-3719-4692-95c7-80082053c0b3 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:50.568535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-2ec39a99-92cd-4528-9d11-98d5bc7d7931 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 08:24:50.568535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:50.568535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:50.568535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:50.568535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:50.568535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:50.568535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:50.568535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:50.568535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:50.568535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:50.568535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:50.568535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:50.568535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:24:50.568535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:24:50.568535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:24:50.568535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:24:50.569622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:50.569622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:50.569622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:24:50.569622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:50.569622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:24:50.569622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:50.569622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:50.569622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:50.569622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:50.569622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:50.569622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:50.569622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:50.569622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:50.569622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:50.569622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:50.569622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:50.569622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:50.569622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:50.570623 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:50.570623 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:50.570623 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:24:50.570623 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:24:50.570623 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:24:50.570623 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:24:50.570623 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:24:50.570623 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:24:50.570623 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 08:24:50.570623 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:50.573102 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 534/1062] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Aug 8 08:24:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:50.573934 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec39a99-92cd-4528-9d11-98d5bc7d7931 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:50.649691 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:50.652717 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92304de4-3719-4692-95c7-80082053c0b3 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:50.671811 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec39a99-92cd-4528-9d11-98d5bc7d7931 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:50.687930 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5f1b4-fce4-4f47-8858-b563ddb43c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:50.688551 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5f1b4-fce4-4f47-8858-b563ddb43c78 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:50.688823 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5f1b4-fce4-4f47-8858-b563ddb43c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:50.689091 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5f1b4-fce4-4f47-8858-b563ddb43c78 None admin] PATH_INFO: `/v3/projects/fe6ee2acdaf2458889009c30ebec29b6` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:50.690217 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c5f1b4-fce4-4f47-8858-b563ddb43c78 None admin] RBAC: Authorizing `identity:get_project(project_id=fe6ee2acdaf2458889009c30ebec29b6)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:50.691626 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c5f1b4-fce4-4f47-8858-b563ddb43c78 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:50.693261 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 529/1063] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:24:50 2022] GET /identity/v3/projects/fe6ee2acdaf2458889009c30ebec29b6 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:50.694004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5f1b4-fce4-4f47-8858-b563ddb43c78 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:50.710297 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a9707-741d-4566-acd1-a646746d7c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:50.710906 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a9707-741d-4566-acd1-a646746d7c9d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:50.711182 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a9707-741d-4566-acd1-a646746d7c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:50.711451 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a9707-741d-4566-acd1-a646746d7c9d None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:50.712074 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a9707-741d-4566-acd1-a646746d7c9d None admin] RBAC: Adding query filter params (type=image) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:50.712372 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a9707-741d-4566-acd1-a646746d7c9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:50.713762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a9707-741d-4566-acd1-a646746d7c9d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:50.719144 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 535/1064] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:50 2022] GET /identity/v3/services?type=image => generated 350 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:50.719752 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a9707-741d-4566-acd1-a646746d7c9d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:50.773675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:50.791618 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5f1b4-fce4-4f47-8858-b563ddb43c78 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:50.821335 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a9707-741d-4566-acd1-a646746d7c9d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:50.825451 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79948c-21b2-44cd-8886-73272bc9c7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:50.826048 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79948c-21b2-44cd-8886-73272bc9c7b0 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:50.826321 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79948c-21b2-44cd-8886-73272bc9c7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:50.826587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79948c-21b2-44cd-8886-73272bc9c7b0 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:50.827765 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff79948c-21b2-44cd-8886-73272bc9c7b0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:50.828071 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff79948c-21b2-44cd-8886-73272bc9c7b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:50.829495 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff79948c-21b2-44cd-8886-73272bc9c7b0 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:50.846507 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 530/1065] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Aug 8 08:24:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:50.847187 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79948c-21b2-44cd-8886-73272bc9c7b0 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:50.859317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63104b33-d797-476a-9043-5a39b14741a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:50.859926 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63104b33-d797-476a-9043-5a39b14741a7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:50.860206 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63104b33-d797-476a-9043-5a39b14741a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:50.860478 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63104b33-d797-476a-9043-5a39b14741a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:50.861229 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63104b33-d797-476a-9043-5a39b14741a7 None admin] RBAC: Adding query filter params (role.id=c1e514a4b5424f959b069f36cbfd8147, scope.project.id=fe6ee2acdaf2458889009c30ebec29b6, user.id=11065c949b954f88becd91e63cb5a81e) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:50.861537 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63104b33-d797-476a-9043-5a39b14741a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:50.862956 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63104b33-d797-476a-9043-5a39b14741a7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:50.868281 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 536/1066] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:24:50 2022] GET /identity/v3/role_assignments?user.id=11065c949b954f88becd91e63cb5a81e&scope.project.id=fe6ee2acdaf2458889009c30ebec29b6&role.id=c1e514a4b5424f959b069f36cbfd8147 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:50.868898 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63104b33-d797-476a-9043-5a39b14741a7 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:50.873129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:50.960843 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79948c-21b2-44cd-8886-73272bc9c7b0 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:50.995526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63104b33-d797-476a-9043-5a39b14741a7 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:51.000975 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42354666-1457-48d7-97e7-550f3a36b5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:51.001631 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42354666-1457-48d7-97e7-550f3a36b5a1 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:51.001907 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42354666-1457-48d7-97e7-550f3a36b5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:51.002178 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42354666-1457-48d7-97e7-550f3a36b5a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:51.002851 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42354666-1457-48d7-97e7-550f3a36b5a1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:51.004234 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42354666-1457-48d7-97e7-550f3a36b5a1 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:51.013465 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:51.032008 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49c202-4137-4d28-928f-87764ecc045b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:51.032574 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49c202-4137-4d28-928f-87764ecc045b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:51.032892 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49c202-4137-4d28-928f-87764ecc045b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:51.033202 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49c202-4137-4d28-928f-87764ecc045b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:51.044320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49c202-4137-4d28-928f-87764ecc045b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:51.045898 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49c202-4137-4d28-928f-87764ecc045b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:51.060097 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-cf49c202-4137-4d28-928f-87764ecc045b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:51.060097 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:51.060097 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:51.060097 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:51.060097 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:51.060097 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:51.060097 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:51.060097 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:51.060097 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:51.060097 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:51.060097 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:51.060097 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:51.060097 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:51.060097 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:51.060097 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:51.060097 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:51.066386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:51.066386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:51.066386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:51.066386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:51.066386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:51.066386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:51.066386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:51.066386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:51.066386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:51.066386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:51.066386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:51.066386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:51.066386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:51.066386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:51.066386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:51.066386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:51.066386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:51.067391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:51.067391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:51.067391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:51.067391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 537/1067] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:51.067391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49c202-4137-4d28-928f-87764ecc045b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:51.085443 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 531/1068] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:24:50 2022] POST /identity/v3/endpoints => generated 327 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:24:51.190003 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49c202-4137-4d28-928f-87764ecc045b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:51.233035 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c0ae04-65b0-4ecc-b9c7-75ff4c5903f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:51.234165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c0ae04-65b0-4ecc-b9c7-75ff4c5903f0 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:51.234609 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c0ae04-65b0-4ecc-b9c7-75ff4c5903f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:51.234888 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c0ae04-65b0-4ecc-b9c7-75ff4c5903f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:51.235532 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c0ae04-65b0-4ecc-b9c7-75ff4c5903f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:51.235861 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c0ae04-65b0-4ecc-b9c7-75ff4c5903f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:51.237300 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c0ae04-65b0-4ecc-b9c7-75ff4c5903f0 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:51.251472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 538/1069] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:51.257482 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42354666-1457-48d7-97e7-550f3a36b5a1 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:51.355087 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:51.373960 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42354666-1457-48d7-97e7-550f3a36b5a1 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:51.415290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bfcee-9397-483e-a720-e4793dccee2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:51.416801 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bfcee-9397-483e-a720-e4793dccee2f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:51.417343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bfcee-9397-483e-a720-e4793dccee2f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:51.418214 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bfcee-9397-483e-a720-e4793dccee2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:51.425271 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5bfcee-9397-483e-a720-e4793dccee2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:51.426697 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5bfcee-9397-483e-a720-e4793dccee2f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:51.438489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-0e5bfcee-9397-483e-a720-e4793dccee2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:51.438489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:51.438489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:51.438489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:51.438489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:51.438489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:51.438489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:51.438489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:51.438489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:51.438489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:51.438489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:51.438489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:51.438489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:24:51.438489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:24:51.438489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:24:51.439462 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:24:51.439462 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:51.439462 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:51.439462 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:51.439462 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:51.439462 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:51.439462 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:51.439462 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:51.439462 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:51.439462 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:51.439462 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:51.439462 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:51.439462 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:51.439462 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:51.439462 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:51.439462 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:51.439462 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:51.440373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:24:51.440373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:24:51.440373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:24:51.440373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:24:51.440373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:24:51.440373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:24:51.440373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:51.440373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:51.440373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 532/1070] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:24:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:51.445739 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c0ae04-65b0-4ecc-b9c7-75ff4c5903f0 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:51.472500 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:51.547147 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c0ae04-65b0-4ecc-b9c7-75ff4c5903f0 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:51.581543 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf381b2-a4a1-4397-be59-d0b1983f5af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:51.582152 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf381b2-a4a1-4397-be59-d0b1983f5af1 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:51.582431 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf381b2-a4a1-4397-be59-d0b1983f5af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:51.582699 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf381b2-a4a1-4397-be59-d0b1983f5af1 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:51.583383 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf381b2-a4a1-4397-be59-d0b1983f5af1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:51.583684 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf381b2-a4a1-4397-be59-d0b1983f5af1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:51.585402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf381b2-a4a1-4397-be59-d0b1983f5af1 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:51.602099 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 539/1071] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:24:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:51.603687 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bfcee-9397-483e-a720-e4793dccee2f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:51.701574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bfcee-9397-483e-a720-e4793dccee2f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:51.735040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88169e8-1828-4714-8647-9e8ccea35e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:51.735691 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88169e8-1828-4714-8647-9e8ccea35e87 None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:51.735967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88169e8-1828-4714-8647-9e8ccea35e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:51.736235 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88169e8-1828-4714-8647-9e8ccea35e87 None admin] PATH_INFO: `/v3/projects/71e5e82c9f454cdd91dfeac5a57968c4/users/a3317f751ec74bde95ec262394001292/roles/b4f3adb27a0048ea86dbf52cb1d2ece5` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:51.738345 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88169e8-1828-4714-8647-9e8ccea35e87 None admin] RBAC: Authorizing `identity:create_grant(project_id=71e5e82c9f454cdd91dfeac5a57968c4, user_id=a3317f751ec74bde95ec262394001292, role_id=b4f3adb27a0048ea86dbf52cb1d2ece5)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:51.739752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88169e8-1828-4714-8647-9e8ccea35e87 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:51.800795 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 533/1072] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 08:24:51 2022] PUT /identity/v3/projects/71e5e82c9f454cdd91dfeac5a57968c4/users/a3317f751ec74bde95ec262394001292/roles/b4f3adb27a0048ea86dbf52cb1d2ece5 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:24:52.872381 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670af3b-6112-47fd-a339-d1a887669c22 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:52.872879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670af3b-6112-47fd-a339-d1a887669c22 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:52.872959 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670af3b-6112-47fd-a339-d1a887669c22 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:52.874519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 540/1073] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:52.879693 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f59ddc-01c8-40bd-9a50-8b1a0b04c6b4 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:52.880110 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f59ddc-01c8-40bd-9a50-8b1a0b04c6b4 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:52.880283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f59ddc-01c8-40bd-9a50-8b1a0b04c6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:52.886878 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:52.900728 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-c5f59ddc-01c8-40bd-9a50-8b1a0b04c6b4 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:52.906964 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c5f59ddc-01c8-40bd-9a50-8b1a0b04c6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:52.970031 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 534/1074] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:52.992428 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670af3b-6112-47fd-a339-d1a887669c22 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:53.077756 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:53.094853 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670af3b-6112-47fd-a339-d1a887669c22 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:53.128923 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea182c6-5999-4063-aaa7-7504e0934b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:53.129507 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea182c6-5999-4063-aaa7-7504e0934b65 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:53.129783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea182c6-5999-4063-aaa7-7504e0934b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:53.130050 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea182c6-5999-4063-aaa7-7504e0934b65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:53.140543 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea182c6-5999-4063-aaa7-7504e0934b65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:24:53.145138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-cea182c6-5999-4063-aaa7-7504e0934b65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:24:53.145138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:53.145138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:53.145138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:53.145138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:53.145138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:53.145138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:53.145138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:53.145138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:53.145138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:53.145138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:53.145138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:53.145138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:24:53.145138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:24:53.145138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:24:53.145138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:24:53.146078 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:24:53.146078 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:24:53.146078 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:24:53.146078 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:24:53.146078 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:53.146078 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:53.146078 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:53.146078 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:53.146078 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:53.146078 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:53.146078 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:53.146078 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:53.146078 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:53.146078 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:53.146078 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:53.146078 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:53.146957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:53.146957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:53.146957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:53.146957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:53.146957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:24:53.146957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:24:53.146957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:24:53.146957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:24:53.146957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:24:53.146957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:24:53.146957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:24:53.146957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:53.149252 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 541/1075] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:24:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:53.152507 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f59ddc-01c8-40bd-9a50-8b1a0b04c6b4 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:53.158479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c264b-33ae-4cae-a698-fa78a580337b None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:53.158769 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c264b-33ae-4cae-a698-fa78a580337b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:53.159044 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c264b-33ae-4cae-a698-fa78a580337b None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:53.160597 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 542/1076] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 08:24:53.165290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6956f6c-3a00-4c7d-b733-60112562c585 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:53.165576 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6956f6c-3a00-4c7d-b733-60112562c585 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:53.165862 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6956f6c-3a00-4c7d-b733-60112562c585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:53.184336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b6956f6c-3a00-4c7d-b733-60112562c585 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:53.185560 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b6956f6c-3a00-4c7d-b733-60112562c585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:53.255300 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f59ddc-01c8-40bd-9a50-8b1a0b04c6b4 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:53.260747 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 543/1077] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 08:24:53.283382 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6956f6c-3a00-4c7d-b733-60112562c585 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:53.287404 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:53.293182 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672436e3-eda5-437a-95c7-3a4d051cbd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:53.293831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672436e3-eda5-437a-95c7-3a4d051cbd50 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:53.294194 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672436e3-eda5-437a-95c7-3a4d051cbd50 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:53.294470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672436e3-eda5-437a-95c7-3a4d051cbd50 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:53.295139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672436e3-eda5-437a-95c7-3a4d051cbd50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:53.297319 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672436e3-eda5-437a-95c7-3a4d051cbd50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:53.297319 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672436e3-eda5-437a-95c7-3a4d051cbd50 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:53.306078 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 535/1078] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:24:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:53.312416 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672436e3-eda5-437a-95c7-3a4d051cbd50 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:53.376431 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:53.403093 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6956f6c-3a00-4c7d-b733-60112562c585 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:53.423656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672436e3-eda5-437a-95c7-3a4d051cbd50 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:53.444991 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690cadbb-3504-4e5c-9efd-a86a905df8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:53.445638 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690cadbb-3504-4e5c-9efd-a86a905df8a8 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:53.445908 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690cadbb-3504-4e5c-9efd-a86a905df8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:53.446173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690cadbb-3504-4e5c-9efd-a86a905df8a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:53.470586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b7cdd-7146-47f5-9ff3-3d8ae7bc7c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:53.471719 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b7cdd-7146-47f5-9ff3-3d8ae7bc7c08 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:53.472148 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b7cdd-7146-47f5-9ff3-3d8ae7bc7c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:53.472655 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b7cdd-7146-47f5-9ff3-3d8ae7bc7c08 None admin] PATH_INFO: `/v3/users/11065c949b954f88becd91e63cb5a81e` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:53.474073 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1b7cdd-7146-47f5-9ff3-3d8ae7bc7c08 None admin] RBAC: Authorizing `identity:get_user(user_id=11065c949b954f88becd91e63cb5a81e)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:53.475629 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1b7cdd-7146-47f5-9ff3-3d8ae7bc7c08 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:53.482256 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 536/1079] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:24:53 2022] GET /identity/v3/users/11065c949b954f88becd91e63cb5a81e => generated 285 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:53.483771 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b7cdd-7146-47f5-9ff3-3d8ae7bc7c08 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:53.517177 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690cadbb-3504-4e5c-9efd-a86a905df8a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:53.518644 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690cadbb-3504-4e5c-9efd-a86a905df8a8 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:53.550711 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:53.591057 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b7cdd-7146-47f5-9ff3-3d8ae7bc7c08 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:53.599450 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:53.626583 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e30bc-3478-4870-9d42-dc845bbb9164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:53.627166 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 544/1080] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:24:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 08:24:53.629536 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e30bc-3478-4870-9d42-dc845bbb9164 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:53.629822 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e30bc-3478-4870-9d42-dc845bbb9164 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:53.630099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e30bc-3478-4870-9d42-dc845bbb9164 None admin] PATH_INFO: `/v3/projects/fe6ee2acdaf2458889009c30ebec29b6` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:53.631261 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e30bc-3478-4870-9d42-dc845bbb9164 None admin] RBAC: Authorizing `identity:get_project(project_id=fe6ee2acdaf2458889009c30ebec29b6)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:53.632690 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e30bc-3478-4870-9d42-dc845bbb9164 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:53.632798 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690cadbb-3504-4e5c-9efd-a86a905df8a8 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:53.639676 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 537/1081] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:24:53 2022] GET /identity/v3/projects/fe6ee2acdaf2458889009c30ebec29b6 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:53.640569 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e30bc-3478-4870-9d42-dc845bbb9164 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:53.742678 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e30bc-3478-4870-9d42-dc845bbb9164 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:53.745399 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690cadbb-3504-4e5c-9efd-a86a905df8a8 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:53.771237 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:53.776948 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7130111-3a6b-4e40-9c89-40938220e78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:53.777668 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7130111-3a6b-4e40-9c89-40938220e78c None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:53.777945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7130111-3a6b-4e40-9c89-40938220e78c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:53.778215 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7130111-3a6b-4e40-9c89-40938220e78c None admin] PATH_INFO: `/v3/projects/fe6ee2acdaf2458889009c30ebec29b6/users/11065c949b954f88becd91e63cb5a81e/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:53.780158 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7130111-3a6b-4e40-9c89-40938220e78c None admin] RBAC: Authorizing `identity:create_grant(project_id=fe6ee2acdaf2458889009c30ebec29b6, user_id=11065c949b954f88becd91e63cb5a81e, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:53.781607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7130111-3a6b-4e40-9c89-40938220e78c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:53.784787 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc1f72d-e9a6-4e0e-9596-5e51ecaa4f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:53.785661 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1f72d-e9a6-4e0e-9596-5e51ecaa4f2d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:53.789261 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1f72d-e9a6-4e0e-9596-5e51ecaa4f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:53.789261 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1f72d-e9a6-4e0e-9596-5e51ecaa4f2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:53.796982 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc1f72d-e9a6-4e0e-9596-5e51ecaa4f2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:53.798454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc1f72d-e9a6-4e0e-9596-5e51ecaa4f2d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:53.813402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-dfc1f72d-e9a6-4e0e-9596-5e51ecaa4f2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:53.813402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:53.813402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:53.813402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:53.813402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:53.813402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:53.813402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:53.813402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:53.813402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:53.813402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:53.813402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:53.813402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:53.813402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:53.813402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:53.813402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:53.813402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:53.814472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:53.814472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:53.814472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:53.814472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:53.814472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:53.814472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:53.814472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:53.814472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:53.814472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:53.814472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:53.814472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:53.814472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:53.814472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:53.814472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:53.814472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:53.814472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:53.814472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:53.815767 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:53.815767 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:53.815767 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:53.815767 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 545/1082] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:24:53.815767 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1488cb-da20-4ada-aa7a-8d4a84a06f74 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:53.816202 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1488cb-da20-4ada-aa7a-8d4a84a06f74 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:53.816202 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1488cb-da20-4ada-aa7a-8d4a84a06f74 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:53.817334 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 546/1083] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:53.820042 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1488cb-da20-4ada-aa7a-8d4a84a06f74 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:53.846471 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 538/1084] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 08:24:53 2022] PUT /identity/v3/projects/fe6ee2acdaf2458889009c30ebec29b6/users/11065c949b954f88becd91e63cb5a81e/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:24:53.847689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a8e81-0e56-4ab5-8a61-9e7bd45f809a None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:53.847972 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a8e81-0e56-4ab5-8a61-9e7bd45f809a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:53.848241 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a8e81-0e56-4ab5-8a61-9e7bd45f809a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:53.853336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:53.873473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-c59a8e81-0e56-4ab5-8a61-9e7bd45f809a None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:53.875224 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c59a8e81-0e56-4ab5-8a61-9e7bd45f809a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:53.951417 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1488cb-da20-4ada-aa7a-8d4a84a06f74 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:53.952575 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 539/1085] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:53.977177 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a8e81-0e56-4ab5-8a61-9e7bd45f809a None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:53.994473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a3982-35d1-4588-8075-d8c187640e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:53.995572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a3982-35d1-4588-8075-d8c187640e03 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:53.996265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a3982-35d1-4588-8075-d8c187640e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:53.996681 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a3982-35d1-4588-8075-d8c187640e03 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:53.997488 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a3982-35d1-4588-8075-d8c187640e03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:53.997923 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a3982-35d1-4588-8075-d8c187640e03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:54.000401 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a3982-35d1-4588-8075-d8c187640e03 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:54.016724 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 547/1086] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:54.093230 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a8e81-0e56-4ab5-8a61-9e7bd45f809a None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:54.116671 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:54.131708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab78b8-b2a7-4d99-b4c0-64e9921d66be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:54.132354 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab78b8-b2a7-4d99-b4c0-64e9921d66be None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:54.132637 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab78b8-b2a7-4d99-b4c0-64e9921d66be None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:54.132905 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab78b8-b2a7-4d99-b4c0-64e9921d66be None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:54.145246 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ab78b8-b2a7-4d99-b4c0-64e9921d66be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:24:54.150080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-f8ab78b8-b2a7-4d99-b4c0-64e9921d66be None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 08:24:54.150080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:54.150080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:54.150080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:54.150080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:54.150080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:54.150080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:54.150080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:54.150080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:54.150080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:54.150080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:54.150080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:54.150080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:24:54.150080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:24:54.150080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:24:54.150080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:24:54.151470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:24:54.151470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:24:54.151470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:24:54.151470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:24:54.151470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:54.151470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:54.151470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:54.151470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:54.151470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:54.151470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:54.151470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:54.151470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:54.151470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:54.151470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:54.151470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:54.151470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:54.152362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:54.152362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:54.152362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:54.152362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:54.152362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:24:54.152362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:24:54.152362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:24:54.152362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:24:54.152362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:24:54.152362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:24:54.152362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 08:24:54.152362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:54.152362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 540/1087] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Aug 8 08:24:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:24:54.160627 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a3982-35d1-4588-8075-d8c187640e03 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:54.247601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:54.265519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a3982-35d1-4588-8075-d8c187640e03 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:54.303567 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e532e-202a-498b-a314-502f21f3fce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:54.304184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e532e-202a-498b-a314-502f21f3fce2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:54.304454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e532e-202a-498b-a314-502f21f3fce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:54.304719 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e532e-202a-498b-a314-502f21f3fce2 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:54.305409 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e532e-202a-498b-a314-502f21f3fce2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:54.305709 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e532e-202a-498b-a314-502f21f3fce2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:54.307071 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e532e-202a-498b-a314-502f21f3fce2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:54.316408 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 548/1088] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Aug 8 08:24:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:54.322160 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab78b8-b2a7-4d99-b4c0-64e9921d66be None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:54.424857 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab78b8-b2a7-4d99-b4c0-64e9921d66be None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:54.457804 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325dbb8-0779-4e19-b2eb-49e9fd06b94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:54.458459 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325dbb8-0779-4e19-b2eb-49e9fd06b94c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:54.458736 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325dbb8-0779-4e19-b2eb-49e9fd06b94c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:54.459008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325dbb8-0779-4e19-b2eb-49e9fd06b94c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:54.470216 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325dbb8-0779-4e19-b2eb-49e9fd06b94c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:54.471659 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325dbb8-0779-4e19-b2eb-49e9fd06b94c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:54.485607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-c325dbb8-0779-4e19-b2eb-49e9fd06b94c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:54.485607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:54.485607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:54.485607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:54.485607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:54.485607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:54.485607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:54.485607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:54.485607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:54.485607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:54.485607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:54.485607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:54.485607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:54.485607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:54.485607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:54.485607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:54.491358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:54.491358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:54.491358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:54.491358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:54.491358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:54.491358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:54.491358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:54.491358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:54.491358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:54.491358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:54.491358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:54.491358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:54.491358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:54.491358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:54.491358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:54.491358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:54.491358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:54.492509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:54.492509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:54.492509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:54.492509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 541/1089] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:54.493399 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e532e-202a-498b-a314-502f21f3fce2 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:54.591148 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e532e-202a-498b-a314-502f21f3fce2 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:54.624930 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b54d9-d815-46fb-bd0e-c6c72dc921ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:54.625561 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b54d9-d815-46fb-bd0e-c6c72dc921ed None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:54.625839 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b54d9-d815-46fb-bd0e-c6c72dc921ed None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:54.626108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b54d9-d815-46fb-bd0e-c6c72dc921ed None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:54.626739 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9b54d9-d815-46fb-bd0e-c6c72dc921ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:54.627038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9b54d9-d815-46fb-bd0e-c6c72dc921ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:54.628404 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9b54d9-d815-46fb-bd0e-c6c72dc921ed None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:54.641196 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 549/1090] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:54.646814 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325dbb8-0779-4e19-b2eb-49e9fd06b94c None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:54.678632 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:54.746075 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325dbb8-0779-4e19-b2eb-49e9fd06b94c None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:54.779001 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155cccc-b07c-4bbd-8e0c-2b53713150bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:54.779613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155cccc-b07c-4bbd-8e0c-2b53713150bc None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:54.779920 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155cccc-b07c-4bbd-8e0c-2b53713150bc None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:54.780191 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155cccc-b07c-4bbd-8e0c-2b53713150bc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:54.787464 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155cccc-b07c-4bbd-8e0c-2b53713150bc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:54.788907 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155cccc-b07c-4bbd-8e0c-2b53713150bc None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:54.800074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-3155cccc-b07c-4bbd-8e0c-2b53713150bc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 08:24:54.800074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:54.800074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:54.800074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:54.800074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:54.800074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:54.800074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:54.800074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:54.800074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:54.800074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:54.800074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:54.800074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:54.800074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:24:54.800074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:24:54.800074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:24:54.800074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:24:54.801005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:54.801005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:54.801005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:24:54.801005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:54.801005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:24:54.801005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:24:54.801005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:54.801005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:54.801005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:54.801005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:54.801005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:54.801005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:54.801005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:54.801005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:54.801005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:54.801005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:54.801005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:54.801005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:54.802008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:54.802008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:54.802008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:24:54.802008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:24:54.802008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:24:54.802008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:24:54.802008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:24:54.802008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:24:54.802008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 08:24:54.802008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:24:54.802008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 542/1091] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Aug 8 08:24:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:54.807039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b54d9-d815-46fb-bd0e-c6c72dc921ed None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:54.818235 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:54.905645 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b54d9-d815-46fb-bd0e-c6c72dc921ed None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:54.939660 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1b9aa-b460-454a-a7d2-f27d57b5716d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:54.940271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1b9aa-b460-454a-a7d2-f27d57b5716d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:54.940546 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1b9aa-b460-454a-a7d2-f27d57b5716d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:54.940814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1b9aa-b460-454a-a7d2-f27d57b5716d None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:54.941567 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e1b9aa-b460-454a-a7d2-f27d57b5716d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:54.941867 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e1b9aa-b460-454a-a7d2-f27d57b5716d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:54.943296 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e1b9aa-b460-454a-a7d2-f27d57b5716d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:54.963316 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 550/1092] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Aug 8 08:24:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:54.969239 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155cccc-b07c-4bbd-8e0c-2b53713150bc None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:55.056467 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:55.065665 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155cccc-b07c-4bbd-8e0c-2b53713150bc None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:55.098659 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b606c2d-6a67-4aee-aa59-691a1ad471b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:55.099251 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b606c2d-6a67-4aee-aa59-691a1ad471b7 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:55.099526 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b606c2d-6a67-4aee-aa59-691a1ad471b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:55.099792 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b606c2d-6a67-4aee-aa59-691a1ad471b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:55.163821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b606c2d-6a67-4aee-aa59-691a1ad471b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:55.165185 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b606c2d-6a67-4aee-aa59-691a1ad471b7 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:55.251065 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:55.260002 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 543/1093] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:24:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 08:24:55.265901 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1b9aa-b460-454a-a7d2-f27d57b5716d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:55.330054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:55.363457 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1b9aa-b460-454a-a7d2-f27d57b5716d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:55.396231 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c60b4f-5755-4423-a1d9-97db2ffdad7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:55.396851 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c60b4f-5755-4423-a1d9-97db2ffdad7e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:55.397126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c60b4f-5755-4423-a1d9-97db2ffdad7e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:55.397426 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c60b4f-5755-4423-a1d9-97db2ffdad7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:55.416346 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c60b4f-5755-4423-a1d9-97db2ffdad7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:55.417838 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c60b4f-5755-4423-a1d9-97db2ffdad7e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:55.432831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-88c60b4f-5755-4423-a1d9-97db2ffdad7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:55.432831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:55.432831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:55.432831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:55.432831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:55.432831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:55.432831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:55.432831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:55.432831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:55.432831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:55.432831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:55.432831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:55.432831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:24:55.432831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:24:55.432831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:24:55.432831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:24:55.433850 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:55.433850 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:55.433850 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:55.433850 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:55.433850 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:55.433850 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:55.433850 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:55.433850 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:55.433850 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:55.433850 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:55.433850 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:55.433850 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:55.433850 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:55.433850 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:55.433850 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:55.433850 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:55.433850 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:24:55.434763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:24:55.434763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:24:55.434763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:55.434763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 551/1094] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:24:55.440510 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b606c2d-6a67-4aee-aa59-691a1ad471b7 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:55.536791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b606c2d-6a67-4aee-aa59-691a1ad471b7 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:55.570093 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ccead-ad84-4e12-b999-eb8f75470e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:55.570698 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ccead-ad84-4e12-b999-eb8f75470e0b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:55.570973 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ccead-ad84-4e12-b999-eb8f75470e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:55.571245 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ccead-ad84-4e12-b999-eb8f75470e0b None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:55.571878 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ccead-ad84-4e12-b999-eb8f75470e0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:55.572176 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ccead-ad84-4e12-b999-eb8f75470e0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:55.573550 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ccead-ad84-4e12-b999-eb8f75470e0b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:55.584929 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 544/1095] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:55.590551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c60b4f-5755-4423-a1d9-97db2ffdad7e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:55.686640 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c60b4f-5755-4423-a1d9-97db2ffdad7e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:55.702486 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:55.719347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27477deb-6335-418e-9e24-e97774294d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:55.719971 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27477deb-6335-418e-9e24-e97774294d63 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:55.720246 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27477deb-6335-418e-9e24-e97774294d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:55.720514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27477deb-6335-418e-9e24-e97774294d63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:55.726984 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27477deb-6335-418e-9e24-e97774294d63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:55.728412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27477deb-6335-418e-9e24-e97774294d63 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:55.740039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-27477deb-6335-418e-9e24-e97774294d63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:55.740039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:55.740039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:55.740039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:55.740039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:55.740039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:55.740039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:55.740039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:55.740039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:55.740039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:55.740039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:55.740039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:55.740039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:24:55.740039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:24:55.740039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:24:55.740950 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:24:55.740950 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:55.740950 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:55.740950 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:55.740950 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:55.740950 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:55.740950 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:55.740950 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:55.740950 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:55.740950 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:55.740950 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:55.740950 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:55.740950 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:55.740950 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:55.740950 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:55.740950 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:55.740950 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:55.741902 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:24:55.741902 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:24:55.741902 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:24:55.741902 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:24:55.741902 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:24:55.741902 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:24:55.741902 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:24:55.741902 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:55.741902 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 552/1096] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:24:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:24:55.746861 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ccead-ad84-4e12-b999-eb8f75470e0b None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:55.799439 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:55.845626 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ccead-ad84-4e12-b999-eb8f75470e0b None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:55.859118 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050064b-fea1-4041-aa6b-e1582f3599ac None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:55.859399 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050064b-fea1-4041-aa6b-e1582f3599ac None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:55.859668 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050064b-fea1-4041-aa6b-e1582f3599ac None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:55.860986 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 553/1097] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 08:24:55.865766 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46502590-d72c-48fc-a26c-25276d0eae33 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:55.866041 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46502590-d72c-48fc-a26c-25276d0eae33 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:55.866306 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46502590-d72c-48fc-a26c-25276d0eae33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:55.878652 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd04d672-57e2-45ca-a8e4-768e264a024f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:55.879253 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04d672-57e2-45ca-a8e4-768e264a024f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:55.879524 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04d672-57e2-45ca-a8e4-768e264a024f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:55.879787 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04d672-57e2-45ca-a8e4-768e264a024f None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:55.880470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd04d672-57e2-45ca-a8e4-768e264a024f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:55.880764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd04d672-57e2-45ca-a8e4-768e264a024f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:55.882185 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd04d672-57e2-45ca-a8e4-768e264a024f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:55.884019 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-46502590-d72c-48fc-a26c-25276d0eae33 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:55.885215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-46502590-d72c-48fc-a26c-25276d0eae33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:55.894406 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 545/1098] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:24:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:55.900254 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd04d672-57e2-45ca-a8e4-768e264a024f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:55.950992 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 554/1099] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 08:24:55.974980 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46502590-d72c-48fc-a26c-25276d0eae33 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:56.000322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd04d672-57e2-45ca-a8e4-768e264a024f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:56.013093 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:56.035084 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d958613-9588-47ac-aedf-799638fe7a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:56.035683 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d958613-9588-47ac-aedf-799638fe7a8e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:56.035955 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d958613-9588-47ac-aedf-799638fe7a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:56.036228 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d958613-9588-47ac-aedf-799638fe7a8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:56.036935 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d958613-9588-47ac-aedf-799638fe7a8e None admin] RBAC: Adding query filter params (role.id=b4f3adb27a0048ea86dbf52cb1d2ece5, scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4, user.id=a3317f751ec74bde95ec262394001292) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:56.037262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d958613-9588-47ac-aedf-799638fe7a8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:56.038657 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d958613-9588-47ac-aedf-799638fe7a8e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:56.044115 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 546/1100] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:24:55 2022] GET /identity/v3/role_assignments?user.id=a3317f751ec74bde95ec262394001292&scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4&role.id=b4f3adb27a0048ea86dbf52cb1d2ece5 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:56.075670 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46502590-d72c-48fc-a26c-25276d0eae33 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:56.086681 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:56.113594 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481904ae-e08f-40e2-9b06-7feaa4a69097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:56.114246 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481904ae-e08f-40e2-9b06-7feaa4a69097 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:56.114518 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481904ae-e08f-40e2-9b06-7feaa4a69097 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:56.114782 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481904ae-e08f-40e2-9b06-7feaa4a69097 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:56.125988 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481904ae-e08f-40e2-9b06-7feaa4a69097 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:24:56.130784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-481904ae-e08f-40e2-9b06-7feaa4a69097 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:24:56.130784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:24:56.130784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:24:56.130784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:24:56.130784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:24:56.130784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:24:56.130784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:24:56.130784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:24:56.130784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:24:56.130784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:24:56.130784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:24:56.130784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:24:56.130784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:24:56.130784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:24:56.130784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:24:56.130784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:24:56.132053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:24:56.132053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:24:56.132053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:24:56.132053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:24:56.132053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:24:56.132053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:24:56.132053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:24:56.132053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:24:56.132053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:24:56.132053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:24:56.132053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:24:56.132053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:24:56.132053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:24:56.132053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:24:56.132053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:24:56.132053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:24:56.132916 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:24:56.132916 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:24:56.132916 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:24:56.132916 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:24:56.132916 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:24:56.132916 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:24:56.132916 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:24:56.132916 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:24:56.132916 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:24:56.132916 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:24:56.132916 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:24:56.132916 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:24:56.132916 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 555/1101] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:24:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:24:56.138277 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d958613-9588-47ac-aedf-799638fe7a8e None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:56.242128 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d958613-9588-47ac-aedf-799638fe7a8e None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:56.276693 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef9dfe6-b230-4511-83e9-4c0d194317c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:56.277374 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9dfe6-b230-4511-83e9-4c0d194317c4 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:56.277653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9dfe6-b230-4511-83e9-4c0d194317c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:56.277925 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9dfe6-b230-4511-83e9-4c0d194317c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:56.278590 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef9dfe6-b230-4511-83e9-4c0d194317c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:56.278894 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef9dfe6-b230-4511-83e9-4c0d194317c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:56.280748 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef9dfe6-b230-4511-83e9-4c0d194317c4 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:56.289560 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 547/1102] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:24:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:24:56.295629 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481904ae-e08f-40e2-9b06-7feaa4a69097 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:56.398610 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481904ae-e08f-40e2-9b06-7feaa4a69097 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:56.436154 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fcb1b1-1301-49e9-a44f-e9ff884b08a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:56.436826 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcb1b1-1301-49e9-a44f-e9ff884b08a3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:56.437103 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcb1b1-1301-49e9-a44f-e9ff884b08a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:56.437408 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcb1b1-1301-49e9-a44f-e9ff884b08a3 None admin] PATH_INFO: `/v3/users/11065c949b954f88becd91e63cb5a81e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:56.438548 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fcb1b1-1301-49e9-a44f-e9ff884b08a3 None admin] RBAC: Authorizing `identity:get_user(user_id=11065c949b954f88becd91e63cb5a81e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:56.439990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fcb1b1-1301-49e9-a44f-e9ff884b08a3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:56.441611 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 556/1103] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:24:56 2022] GET /identity/v3/users/11065c949b954f88becd91e63cb5a81e => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:56.447692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef9dfe6-b230-4511-83e9-4c0d194317c4 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:56.489927 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:56.548373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef9dfe6-b230-4511-83e9-4c0d194317c4 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:56.581756 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752168b6-36dc-4094-a189-a70a7eebc871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:56.582357 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752168b6-36dc-4094-a189-a70a7eebc871 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:56.582632 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752168b6-36dc-4094-a189-a70a7eebc871 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:56.582898 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752168b6-36dc-4094-a189-a70a7eebc871 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:56.651656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752168b6-36dc-4094-a189-a70a7eebc871 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:56.653104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752168b6-36dc-4094-a189-a70a7eebc871 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:56.692617 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:56.753438 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 548/1104] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:24:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 08:24:56.759714 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fcb1b1-1301-49e9-a44f-e9ff884b08a3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:56.796542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:56.862202 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fcb1b1-1301-49e9-a44f-e9ff884b08a3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:56.896690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296cb51b-cf57-46d6-b6ec-22537b0a855b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:56.897355 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296cb51b-cf57-46d6-b6ec-22537b0a855b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:56.897632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296cb51b-cf57-46d6-b6ec-22537b0a855b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:56.897901 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296cb51b-cf57-46d6-b6ec-22537b0a855b None admin] PATH_INFO: `/v3/projects/fe6ee2acdaf2458889009c30ebec29b6` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:56.899002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296cb51b-cf57-46d6-b6ec-22537b0a855b None admin] RBAC: Authorizing `identity:get_project(project_id=fe6ee2acdaf2458889009c30ebec29b6)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:56.900422 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296cb51b-cf57-46d6-b6ec-22537b0a855b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:56.902104 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 557/1105] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:24:56 2022] GET /identity/v3/projects/fe6ee2acdaf2458889009c30ebec29b6 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:56.908406 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752168b6-36dc-4094-a189-a70a7eebc871 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:57.004234 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752168b6-36dc-4094-a189-a70a7eebc871 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:57.036687 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086cf44-56f8-45c6-b186-5c102e181031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:57.037299 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086cf44-56f8-45c6-b186-5c102e181031 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:57.037578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086cf44-56f8-45c6-b186-5c102e181031 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:57.037850 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086cf44-56f8-45c6-b186-5c102e181031 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:57.038557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8086cf44-56f8-45c6-b186-5c102e181031 None admin] RBAC: Adding query filter params (role.id=c1e514a4b5424f959b069f36cbfd8147, scope.project.id=fe6ee2acdaf2458889009c30ebec29b6, user.id=11065c949b954f88becd91e63cb5a81e) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:24:57.038861 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8086cf44-56f8-45c6-b186-5c102e181031 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:57.040263 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8086cf44-56f8-45c6-b186-5c102e181031 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:57.046128 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 549/1106] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:24:56 2022] GET /identity/v3/role_assignments?user.id=11065c949b954f88becd91e63cb5a81e&scope.project.id=fe6ee2acdaf2458889009c30ebec29b6&role.id=c1e514a4b5424f959b069f36cbfd8147 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:59.060630 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2358e4-48a6-440b-8525-8e1eaaa87a57 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:59.061432 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2358e4-48a6-440b-8525-8e1eaaa87a57 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:59.061432 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2358e4-48a6-440b-8525-8e1eaaa87a57 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:59.062876 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 558/1107] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:24:59.067996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c8a3a-b462-4bcc-8ee0-372bb597bfff None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:59.068406 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c8a3a-b462-4bcc-8ee0-372bb597bfff None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:59.068547 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c8a3a-b462-4bcc-8ee0-372bb597bfff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:59.074753 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:59.087506 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-6b4c8a3a-b462-4bcc-8ee0-372bb597bfff None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:24:59.088819 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6b4c8a3a-b462-4bcc-8ee0-372bb597bfff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:24:59.152244 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 550/1108] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:24:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:24:59.176661 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2358e4-48a6-440b-8525-8e1eaaa87a57 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:59.275932 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2358e4-48a6-440b-8525-8e1eaaa87a57 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:59.278212 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:59.312918 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad878bc-7777-4db0-84a3-a86f162db201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:59.313565 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad878bc-7777-4db0-84a3-a86f162db201 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:59.314094 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad878bc-7777-4db0-84a3-a86f162db201 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:59.314528 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad878bc-7777-4db0-84a3-a86f162db201 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:59.315871 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad878bc-7777-4db0-84a3-a86f162db201 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:59.317341 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad878bc-7777-4db0-84a3-a86f162db201 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:59.318907 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 559/1109] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:24:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:24:59.324647 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c8a3a-b462-4bcc-8ee0-372bb597bfff None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:24:59.419746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c8a3a-b462-4bcc-8ee0-372bb597bfff None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:24:59.448430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:24:59.454208 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5836275-d56d-4a88-8718-c10a1bd4ad6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:24:59.454886 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5836275-d56d-4a88-8718-c10a1bd4ad6c None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:24:59.455166 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5836275-d56d-4a88-8718-c10a1bd4ad6c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:24:59.455439 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5836275-d56d-4a88-8718-c10a1bd4ad6c None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:24:59.456079 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5836275-d56d-4a88-8718-c10a1bd4ad6c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:24:59.457602 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5836275-d56d-4a88-8718-c10a1bd4ad6c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:24:59.571421 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 551/1110] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 08:24:59 2022] POST /identity/v3/users => generated 286 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:25:01.570153 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ef99c-b864-47d8-9dfc-cd67bf6a8b8f None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:01.570665 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ef99c-b864-47d8-9dfc-cd67bf6a8b8f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:01.570665 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ef99c-b864-47d8-9dfc-cd67bf6a8b8f None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:01.571996 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 560/1111] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:25:01.577634 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587db13c-5997-4d5d-bc4b-06aa8a32b62c None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:01.578059 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587db13c-5997-4d5d-bc4b-06aa8a32b62c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:01.578167 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587db13c-5997-4d5d-bc4b-06aa8a32b62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:01.595825 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-587db13c-5997-4d5d-bc4b-06aa8a32b62c None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:25:01.596966 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-587db13c-5997-4d5d-bc4b-06aa8a32b62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:25:01.662992 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 552/1112] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:25:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:25:01.685703 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ef99c-b864-47d8-9dfc-cd67bf6a8b8f None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:01.783053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ef99c-b864-47d8-9dfc-cd67bf6a8b8f None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:01.815365 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c00857-fefc-4017-a3df-fef73ac25c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:01.815978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c00857-fefc-4017-a3df-fef73ac25c08 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:01.816257 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c00857-fefc-4017-a3df-fef73ac25c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:01.816528 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c00857-fefc-4017-a3df-fef73ac25c08 None admin] PATH_INFO: `/v3/roles/152f4082bb67479ebaeb08c2fa683399` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:01.817923 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c00857-fefc-4017-a3df-fef73ac25c08 None admin] RBAC: Authorizing `identity:get_role(role_id=152f4082bb67479ebaeb08c2fa683399)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:01.819286 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c00857-fefc-4017-a3df-fef73ac25c08 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:01.820563 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 561/1113] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:25:01 2022] GET /identity/v3/roles/152f4082bb67479ebaeb08c2fa683399 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:01.826301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587db13c-5997-4d5d-bc4b-06aa8a32b62c None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:01.922225 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587db13c-5997-4d5d-bc4b-06aa8a32b62c None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:01.940008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:01.956046 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97430f8f-fc8c-4863-b2c3-0c3fbe1acfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:01.956664 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97430f8f-fc8c-4863-b2c3-0c3fbe1acfeb None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:01.956943 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97430f8f-fc8c-4863-b2c3-0c3fbe1acfeb None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:01.957265 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97430f8f-fc8c-4863-b2c3-0c3fbe1acfeb None admin] PATH_INFO: `/v3/users/13eaf55f25e34693ad3e61e9f16f9923` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:01.971174 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97430f8f-fc8c-4863-b2c3-0c3fbe1acfeb None admin] RBAC: Authorizing `identity:get_user(user_id=13eaf55f25e34693ad3e61e9f16f9923)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:01.972631 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97430f8f-fc8c-4863-b2c3-0c3fbe1acfeb None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:01.974258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 553/1114] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:25:01 2022] GET /identity/v3/users/13eaf55f25e34693ad3e61e9f16f9923 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:25:01.979965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c00857-fefc-4017-a3df-fef73ac25c08 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:02.020850 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:02.076635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c00857-fefc-4017-a3df-fef73ac25c08 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:02.111030 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea34801-ddee-4bb9-9899-9f59b8fadec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:02.111786 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea34801-ddee-4bb9-9899-9f59b8fadec6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:02.112066 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea34801-ddee-4bb9-9899-9f59b8fadec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:02.112338 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea34801-ddee-4bb9-9899-9f59b8fadec6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:02.176279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea34801-ddee-4bb9-9899-9f59b8fadec6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:02.177748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea34801-ddee-4bb9-9899-9f59b8fadec6 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:02.216328 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:02.272634 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 562/1115] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:25:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 08:25:02.278543 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97430f8f-fc8c-4863-b2c3-0c3fbe1acfeb None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:02.374394 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97430f8f-fc8c-4863-b2c3-0c3fbe1acfeb None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:02.406486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868a2c1-0bed-4382-876f-8d889befbf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:02.407097 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868a2c1-0bed-4382-876f-8d889befbf89 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:02.407396 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868a2c1-0bed-4382-876f-8d889befbf89 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:02.407669 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868a2c1-0bed-4382-876f-8d889befbf89 None admin] PATH_INFO: `/v3/projects/fe6ee2acdaf2458889009c30ebec29b6` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:02.408783 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868a2c1-0bed-4382-876f-8d889befbf89 None admin] RBAC: Authorizing `identity:get_project(project_id=fe6ee2acdaf2458889009c30ebec29b6)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:02.410230 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868a2c1-0bed-4382-876f-8d889befbf89 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:02.411870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 554/1116] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:25:02 2022] GET /identity/v3/projects/fe6ee2acdaf2458889009c30ebec29b6 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:02.417956 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea34801-ddee-4bb9-9899-9f59b8fadec6 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:02.512930 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea34801-ddee-4bb9-9899-9f59b8fadec6 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:02.545633 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b077ee-2fc6-4e1b-bcad-b199aea1323f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:02.546268 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b077ee-2fc6-4e1b-bcad-b199aea1323f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:02.546550 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b077ee-2fc6-4e1b-bcad-b199aea1323f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:02.546823 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b077ee-2fc6-4e1b-bcad-b199aea1323f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:02.547546 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b077ee-2fc6-4e1b-bcad-b199aea1323f None admin] RBAC: Adding query filter params (role.id=152f4082bb67479ebaeb08c2fa683399, scope.project.id=fe6ee2acdaf2458889009c30ebec29b6, user.id=13eaf55f25e34693ad3e61e9f16f9923) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:25:02.547851 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b077ee-2fc6-4e1b-bcad-b199aea1323f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:02.549299 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b077ee-2fc6-4e1b-bcad-b199aea1323f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:02.554596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 563/1117] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:25:02 2022] GET /identity/v3/role_assignments?user.id=13eaf55f25e34693ad3e61e9f16f9923&scope.project.id=fe6ee2acdaf2458889009c30ebec29b6&role.id=152f4082bb67479ebaeb08c2fa683399 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:04.498177 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ba4b1-54d2-4e6d-80b5-b774b7b462b9 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:04.499339 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ba4b1-54d2-4e6d-80b5-b774b7b462b9 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:04.499339 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ba4b1-54d2-4e6d-80b5-b774b7b462b9 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:04.500682 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 555/1118] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:25:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:25:04.505896 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584debcd-cdf5-4bb9-80ba-d89a5033d17c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:04.506284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584debcd-cdf5-4bb9-80ba-d89a5033d17c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:04.506446 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584debcd-cdf5-4bb9-80ba-d89a5033d17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:04.513589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:04.524024 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-584debcd-cdf5-4bb9-80ba-d89a5033d17c None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:25:04.525209 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-584debcd-cdf5-4bb9-80ba-d89a5033d17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:25:04.590031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 564/1119] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:25:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:25:04.613626 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ba4b1-54d2-4e6d-80b5-b774b7b462b9 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:04.658814 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:04.711336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ba4b1-54d2-4e6d-80b5-b774b7b462b9 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:04.743479 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334fff7-6b1d-4fc0-8e45-8cd0d149428d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:04.744082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334fff7-6b1d-4fc0-8e45-8cd0d149428d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:04.744358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334fff7-6b1d-4fc0-8e45-8cd0d149428d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:04.744627 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334fff7-6b1d-4fc0-8e45-8cd0d149428d None admin] PATH_INFO: `/v3/roles/152f4082bb67479ebaeb08c2fa683399` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:04.746016 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334fff7-6b1d-4fc0-8e45-8cd0d149428d None admin] RBAC: Authorizing `identity:get_role(role_id=152f4082bb67479ebaeb08c2fa683399)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:04.747362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334fff7-6b1d-4fc0-8e45-8cd0d149428d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:04.749378 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 556/1120] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:25:04 2022] GET /identity/v3/roles/152f4082bb67479ebaeb08c2fa683399 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:04.755271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584debcd-cdf5-4bb9-80ba-d89a5033d17c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:04.851355 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584debcd-cdf5-4bb9-80ba-d89a5033d17c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:04.883596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21efb9a1-4949-4263-9cbe-fdd45f328ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:04.884206 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21efb9a1-4949-4263-9cbe-fdd45f328ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:04.884481 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21efb9a1-4949-4263-9cbe-fdd45f328ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:04.884750 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21efb9a1-4949-4263-9cbe-fdd45f328ec7 None admin] PATH_INFO: `/v3/users/13eaf55f25e34693ad3e61e9f16f9923` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:04.885860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21efb9a1-4949-4263-9cbe-fdd45f328ec7 None admin] RBAC: Authorizing `identity:get_user(user_id=13eaf55f25e34693ad3e61e9f16f9923)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:04.887274 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21efb9a1-4949-4263-9cbe-fdd45f328ec7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:04.888799 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 565/1121] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:25:04 2022] GET /identity/v3/users/13eaf55f25e34693ad3e61e9f16f9923 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:04.894913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334fff7-6b1d-4fc0-8e45-8cd0d149428d None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:04.991042 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334fff7-6b1d-4fc0-8e45-8cd0d149428d None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:05.003602 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:05.024081 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8006b5-acfb-49cb-bfd0-8357b5c89c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:05.024692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8006b5-acfb-49cb-bfd0-8357b5c89c0b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:05.024971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8006b5-acfb-49cb-bfd0-8357b5c89c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:05.025273 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8006b5-acfb-49cb-bfd0-8357b5c89c0b None admin] PATH_INFO: `/v3/projects/fe6ee2acdaf2458889009c30ebec29b6` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:05.026364 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8006b5-acfb-49cb-bfd0-8357b5c89c0b None admin] RBAC: Authorizing `identity:get_project(project_id=fe6ee2acdaf2458889009c30ebec29b6)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:05.027772 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8006b5-acfb-49cb-bfd0-8357b5c89c0b None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:05.029403 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 557/1122] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:25:04 2022] GET /identity/v3/projects/fe6ee2acdaf2458889009c30ebec29b6 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:05.035447 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21efb9a1-4949-4263-9cbe-fdd45f328ec7 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:05.038279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:05.146757 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21efb9a1-4949-4263-9cbe-fdd45f328ec7 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:05.178788 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de9e35-a91f-42fd-b927-2980de06bd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:05.179428 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de9e35-a91f-42fd-b927-2980de06bd54 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:05.179704 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de9e35-a91f-42fd-b927-2980de06bd54 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:05.179973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de9e35-a91f-42fd-b927-2980de06bd54 None admin] PATH_INFO: `/v3/projects/fe6ee2acdaf2458889009c30ebec29b6/users/13eaf55f25e34693ad3e61e9f16f9923/roles/152f4082bb67479ebaeb08c2fa683399` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:05.182083 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de9e35-a91f-42fd-b927-2980de06bd54 None admin] RBAC: Authorizing `identity:create_grant(project_id=fe6ee2acdaf2458889009c30ebec29b6, user_id=13eaf55f25e34693ad3e61e9f16f9923, role_id=152f4082bb67479ebaeb08c2fa683399)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:05.183492 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de9e35-a91f-42fd-b927-2980de06bd54 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:05.290853 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 566/1123] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 08:25:05 2022] PUT /identity/v3/projects/fe6ee2acdaf2458889009c30ebec29b6/users/13eaf55f25e34693ad3e61e9f16f9923/roles/152f4082bb67479ebaeb08c2fa683399 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:25:07.237400 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36dba4-5bf1-4ff8-bb31-68ca2c97f9a3 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:07.237958 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36dba4-5bf1-4ff8-bb31-68ca2c97f9a3 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:07.237958 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36dba4-5bf1-4ff8-bb31-68ca2c97f9a3 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:07.239291 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 558/1124] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:25:07.244320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501be15-4521-4eac-8e8d-c57477d32e88 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:07.244767 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501be15-4521-4eac-8e8d-c57477d32e88 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:07.244874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501be15-4521-4eac-8e8d-c57477d32e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:07.263612 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-9501be15-4521-4eac-8e8d-c57477d32e88 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:25:07.265483 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9501be15-4521-4eac-8e8d-c57477d32e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:25:07.318773 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:07.329302 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 567/1125] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:25:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:25:07.353139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36dba4-5bf1-4ff8-bb31-68ca2c97f9a3 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:07.450654 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36dba4-5bf1-4ff8-bb31-68ca2c97f9a3 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:07.483809 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9569751d-16a8-43cf-ac07-c4c500f1fe90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:07.484415 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569751d-16a8-43cf-ac07-c4c500f1fe90 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:07.484690 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569751d-16a8-43cf-ac07-c4c500f1fe90 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:07.484959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569751d-16a8-43cf-ac07-c4c500f1fe90 None admin] PATH_INFO: `/v3/roles/152f4082bb67479ebaeb08c2fa683399` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:07.486340 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9569751d-16a8-43cf-ac07-c4c500f1fe90 None admin] RBAC: Authorizing `identity:get_role(role_id=152f4082bb67479ebaeb08c2fa683399)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:07.487736 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9569751d-16a8-43cf-ac07-c4c500f1fe90 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:07.488989 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 559/1126] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:25:07 2022] GET /identity/v3/roles/152f4082bb67479ebaeb08c2fa683399 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:07.494825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501be15-4521-4eac-8e8d-c57477d32e88 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:07.589791 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501be15-4521-4eac-8e8d-c57477d32e88 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:07.621788 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f405ae9e-6901-4ed5-bf7d-445008f7a9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:07.622408 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405ae9e-6901-4ed5-bf7d-445008f7a9e8 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:07.622683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405ae9e-6901-4ed5-bf7d-445008f7a9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:07.622952 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405ae9e-6901-4ed5-bf7d-445008f7a9e8 None admin] PATH_INFO: `/v3/users/13eaf55f25e34693ad3e61e9f16f9923` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:07.624570 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f405ae9e-6901-4ed5-bf7d-445008f7a9e8 None admin] RBAC: Authorizing `identity:get_user(user_id=13eaf55f25e34693ad3e61e9f16f9923)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:07.626078 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f405ae9e-6901-4ed5-bf7d-445008f7a9e8 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:07.627653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 568/1127] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:25:07 2022] GET /identity/v3/users/13eaf55f25e34693ad3e61e9f16f9923 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:07.633404 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9569751d-16a8-43cf-ac07-c4c500f1fe90 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:07.681888 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:07.730092 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9569751d-16a8-43cf-ac07-c4c500f1fe90 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:07.762378 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702ed34-bd1c-44b6-a4dd-09048d780957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:07.762907 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702ed34-bd1c-44b6-a4dd-09048d780957 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:07.763181 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702ed34-bd1c-44b6-a4dd-09048d780957 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:07.763449 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702ed34-bd1c-44b6-a4dd-09048d780957 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:07.827163 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702ed34-bd1c-44b6-a4dd-09048d780957 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:07.828575 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702ed34-bd1c-44b6-a4dd-09048d780957 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:07.890698 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:07.936090 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 560/1128] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:25:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 08:25:07.942198 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f405ae9e-6901-4ed5-bf7d-445008f7a9e8 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:08.001569 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:08.042392 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f405ae9e-6901-4ed5-bf7d-445008f7a9e8 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:08.078544 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f984e7b-c64c-4975-9b74-874e6189a1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:08.079177 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f984e7b-c64c-4975-9b74-874e6189a1dc None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:08.079452 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f984e7b-c64c-4975-9b74-874e6189a1dc None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:08.079721 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f984e7b-c64c-4975-9b74-874e6189a1dc None admin] PATH_INFO: `/v3/projects/fe6ee2acdaf2458889009c30ebec29b6` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:08.080833 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f984e7b-c64c-4975-9b74-874e6189a1dc None admin] RBAC: Authorizing `identity:get_project(project_id=fe6ee2acdaf2458889009c30ebec29b6)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:08.082311 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f984e7b-c64c-4975-9b74-874e6189a1dc None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:08.083928 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 569/1129] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:25:07 2022] GET /identity/v3/projects/fe6ee2acdaf2458889009c30ebec29b6 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:08.090544 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702ed34-bd1c-44b6-a4dd-09048d780957 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:08.188972 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702ed34-bd1c-44b6-a4dd-09048d780957 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:08.222048 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe670b7-46b2-49ce-a3ab-e9806506318e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:08.222638 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe670b7-46b2-49ce-a3ab-e9806506318e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:08.222916 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe670b7-46b2-49ce-a3ab-e9806506318e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:08.223184 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe670b7-46b2-49ce-a3ab-e9806506318e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:08.223897 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe670b7-46b2-49ce-a3ab-e9806506318e None admin] RBAC: Adding query filter params (role.id=152f4082bb67479ebaeb08c2fa683399, scope.project.id=fe6ee2acdaf2458889009c30ebec29b6, user.id=13eaf55f25e34693ad3e61e9f16f9923) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:25:08.224201 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe670b7-46b2-49ce-a3ab-e9806506318e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:08.225651 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe670b7-46b2-49ce-a3ab-e9806506318e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:08.231104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 561/1130] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:25:08 2022] GET /identity/v3/role_assignments?user.id=13eaf55f25e34693ad3e61e9f16f9923&scope.project.id=fe6ee2acdaf2458889009c30ebec29b6&role.id=152f4082bb67479ebaeb08c2fa683399 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:10.229876 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c76a55-b2a1-4ed9-853f-9fe66d874558 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:10.230493 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c76a55-b2a1-4ed9-853f-9fe66d874558 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:10.230493 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c76a55-b2a1-4ed9-853f-9fe66d874558 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:10.231761 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 570/1131] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:25:10.237844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6361dd-cc1f-4e02-abec-fceece54c967 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:10.238351 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6361dd-cc1f-4e02-abec-fceece54c967 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:10.238459 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6361dd-cc1f-4e02-abec-fceece54c967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:10.257373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-8c6361dd-cc1f-4e02-abec-fceece54c967 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:25:10.258620 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8c6361dd-cc1f-4e02-abec-fceece54c967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:25:10.266941 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:10.324909 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 562/1132] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:25:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 08:25:10.348914 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c76a55-b2a1-4ed9-853f-9fe66d874558 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:10.445493 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c76a55-b2a1-4ed9-853f-9fe66d874558 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:10.471574 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:10.479000 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ccb12b-dab7-49eb-9992-40c8aed335b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:10.479620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccb12b-dab7-49eb-9992-40c8aed335b4 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:10.485209 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccb12b-dab7-49eb-9992-40c8aed335b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:10.485209 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccb12b-dab7-49eb-9992-40c8aed335b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:10.485209 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ccb12b-dab7-49eb-9992-40c8aed335b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:10.485209 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ccb12b-dab7-49eb-9992-40c8aed335b4 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:10.485209 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 571/1133] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:25:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:10.490491 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6361dd-cc1f-4e02-abec-fceece54c967 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:10.586397 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6361dd-cc1f-4e02-abec-fceece54c967 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:10.618449 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45b6b8-c98e-452c-bfc9-114147c4ea7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:10.619034 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45b6b8-c98e-452c-bfc9-114147c4ea7e None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:10.619310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45b6b8-c98e-452c-bfc9-114147c4ea7e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:10.619579 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45b6b8-c98e-452c-bfc9-114147c4ea7e None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:10.620214 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45b6b8-c98e-452c-bfc9-114147c4ea7e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:10.621662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45b6b8-c98e-452c-bfc9-114147c4ea7e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:10.635802 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:10.726900 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 563/1134] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Aug 8 08:25:10 2022] POST /identity/v3/projects => generated 311 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:25:12.727050 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd82ac-2595-4a3c-ad6d-e65893e73b03 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:12.727563 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd82ac-2595-4a3c-ad6d-e65893e73b03 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:12.727563 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd82ac-2595-4a3c-ad6d-e65893e73b03 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:12.728823 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 572/1135] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:25:12.733832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf218dd-040f-4a8b-a9fc-a8180465e459 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:12.734251 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf218dd-040f-4a8b-a9fc-a8180465e459 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:12.734384 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf218dd-040f-4a8b-a9fc-a8180465e459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:12.752542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-8bf218dd-040f-4a8b-a9fc-a8180465e459 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:25:12.753746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8bf218dd-040f-4a8b-a9fc-a8180465e459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:25:12.816976 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 564/1136] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:25:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:25:12.840964 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd82ac-2595-4a3c-ad6d-e65893e73b03 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:12.939018 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd82ac-2595-4a3c-ad6d-e65893e73b03 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:12.972316 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3f0d3-ca51-4d5a-be79-4b80fc51a630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:12.972936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3f0d3-ca51-4d5a-be79-4b80fc51a630 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:12.973246 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3f0d3-ca51-4d5a-be79-4b80fc51a630 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:12.973522 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3f0d3-ca51-4d5a-be79-4b80fc51a630 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:12.974625 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3f0d3-ca51-4d5a-be79-4b80fc51a630 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:12.976077 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3f0d3-ca51-4d5a-be79-4b80fc51a630 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:12.977710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 573/1137] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:25:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:12.984510 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf218dd-040f-4a8b-a9fc-a8180465e459 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:13.079430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf218dd-040f-4a8b-a9fc-a8180465e459 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:13.103627 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:13.114371 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc16e86-428e-4e08-9e9a-389782d3e506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:13.115978 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc16e86-428e-4e08-9e9a-389782d3e506 None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:13.115978 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc16e86-428e-4e08-9e9a-389782d3e506 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:13.115978 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc16e86-428e-4e08-9e9a-389782d3e506 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:13.116220 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc16e86-428e-4e08-9e9a-389782d3e506 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:13.117595 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc16e86-428e-4e08-9e9a-389782d3e506 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:13.212176 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 565/1138] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 08:25:12 2022] POST /identity/v3/users => generated 286 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:25:15.251043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa77d7-7d07-4406-9709-c2c73021ad13 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:15.251591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa77d7-7d07-4406-9709-c2c73021ad13 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:15.251591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa77d7-7d07-4406-9709-c2c73021ad13 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:15.252905 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 574/1139] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:25:15.258124 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff58339-6a1c-4125-b621-1eb05901c4fa None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:15.258525 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff58339-6a1c-4125-b621-1eb05901c4fa None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:15.258667 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff58339-6a1c-4125-b621-1eb05901c4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:15.276748 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-5ff58339-6a1c-4125-b621-1eb05901c4fa None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:25:15.277943 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5ff58339-6a1c-4125-b621-1eb05901c4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:25:15.341558 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 566/1140] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:25:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:25:15.364570 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa77d7-7d07-4406-9709-c2c73021ad13 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:15.425864 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:15.463228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa77d7-7d07-4406-9709-c2c73021ad13 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:15.495630 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7d67a-d05c-4f2e-b011-e70ab15a94d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:15.496248 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7d67a-d05c-4f2e-b011-e70ab15a94d6 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:15.496526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7d67a-d05c-4f2e-b011-e70ab15a94d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:15.496795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7d67a-d05c-4f2e-b011-e70ab15a94d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:15.507246 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c7d67a-d05c-4f2e-b011-e70ab15a94d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:25:15.511814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-91c7d67a-d05c-4f2e-b011-e70ab15a94d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:25:15.511814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:25:15.511814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:25:15.511814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:25:15.511814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:25:15.511814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:25:15.511814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:25:15.511814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:25:15.511814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:25:15.511814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:25:15.511814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:25:15.511814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:25:15.511814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:25:15.511814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:25:15.511814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:25:15.511814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:25:15.512948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:25:15.512948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:25:15.512948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:25:15.512948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:25:15.512948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:25:15.512948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:25:15.512948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:25:15.512948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:25:15.512948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:25:15.512948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:25:15.512948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:25:15.512948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:25:15.512948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:25:15.512948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:25:15.512948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:25:15.512948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:25:15.514058 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:25:15.514058 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:25:15.514058 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:25:15.514058 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:25:15.514058 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:25:15.514058 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:25:15.514058 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:25:15.514058 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:25:15.514058 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:25:15.514058 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:25:15.514058 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:25:15.514058 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:25:15.514058 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 575/1141] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:25:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:25:15.519404 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff58339-6a1c-4125-b621-1eb05901c4fa None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:15.613737 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff58339-6a1c-4125-b621-1eb05901c4fa None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:15.624455 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:15.646399 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1737da47-eb3d-49a9-8a3b-c9231306661a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:15.646992 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737da47-eb3d-49a9-8a3b-c9231306661a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:15.647270 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737da47-eb3d-49a9-8a3b-c9231306661a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:15.647539 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737da47-eb3d-49a9-8a3b-c9231306661a None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:15.648195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1737da47-eb3d-49a9-8a3b-c9231306661a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:25:15.648490 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1737da47-eb3d-49a9-8a3b-c9231306661a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:15.649874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1737da47-eb3d-49a9-8a3b-c9231306661a None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:15.657997 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 567/1142] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:25:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:15.663274 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7d67a-d05c-4f2e-b011-e70ab15a94d6 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:15.761542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7d67a-d05c-4f2e-b011-e70ab15a94d6 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:15.784396 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:15.794546 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c931ba1-00f4-48d3-9319-b9a5c7757119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:15.795168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c931ba1-00f4-48d3-9319-b9a5c7757119 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:15.796039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c931ba1-00f4-48d3-9319-b9a5c7757119 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:15.796314 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c931ba1-00f4-48d3-9319-b9a5c7757119 None admin] PATH_INFO: `/v3/users/a0f66dbcf967418594605d16a971f253` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:15.808887 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c931ba1-00f4-48d3-9319-b9a5c7757119 None admin] RBAC: Authorizing `identity:get_user(user_id=a0f66dbcf967418594605d16a971f253)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:15.810352 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c931ba1-00f4-48d3-9319-b9a5c7757119 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:15.812601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 576/1143] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:25:15 2022] GET /identity/v3/users/a0f66dbcf967418594605d16a971f253 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:15.817970 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1737da47-eb3d-49a9-8a3b-c9231306661a None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:15.917889 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1737da47-eb3d-49a9-8a3b-c9231306661a None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:15.950268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebf311-41b5-435e-8daf-3415f567dd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:15.950858 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebf311-41b5-435e-8daf-3415f567dd60 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:15.951132 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebf311-41b5-435e-8daf-3415f567dd60 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:15.951401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebf311-41b5-435e-8daf-3415f567dd60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:15.991047 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:16.016098 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ebf311-41b5-435e-8daf-3415f567dd60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:16.017483 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ebf311-41b5-435e-8daf-3415f567dd60 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:16.113107 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 568/1144] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:25:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 08:25:16.119021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c931ba1-00f4-48d3-9319-b9a5c7757119 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:16.213587 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c931ba1-00f4-48d3-9319-b9a5c7757119 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:16.247017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf958202-b68b-4423-aecb-ea6a7772ef70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:16.247637 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf958202-b68b-4423-aecb-ea6a7772ef70 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:16.247916 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf958202-b68b-4423-aecb-ea6a7772ef70 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:16.248189 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf958202-b68b-4423-aecb-ea6a7772ef70 None admin] PATH_INFO: `/v3/projects/16fb5037512748fa825f494a820ae384` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:16.249392 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf958202-b68b-4423-aecb-ea6a7772ef70 None admin] RBAC: Authorizing `identity:get_project(project_id=16fb5037512748fa825f494a820ae384)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:16.250809 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf958202-b68b-4423-aecb-ea6a7772ef70 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:16.252400 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 577/1145] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:25:16 2022] GET /identity/v3/projects/16fb5037512748fa825f494a820ae384 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:16.258440 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebf311-41b5-435e-8daf-3415f567dd60 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:16.324514 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:16.352693 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebf311-41b5-435e-8daf-3415f567dd60 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:16.384681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91a056-8604-4bd4-b614-69f2f6e236f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:16.385324 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91a056-8604-4bd4-b614-69f2f6e236f2 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:16.385603 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91a056-8604-4bd4-b614-69f2f6e236f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:16.385872 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91a056-8604-4bd4-b614-69f2f6e236f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:16.386578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a91a056-8604-4bd4-b614-69f2f6e236f2 None admin] RBAC: Adding query filter params (role.id=c1e514a4b5424f959b069f36cbfd8147, scope.project.id=16fb5037512748fa825f494a820ae384, user.id=a0f66dbcf967418594605d16a971f253) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:25:16.386879 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a91a056-8604-4bd4-b614-69f2f6e236f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:16.388274 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a91a056-8604-4bd4-b614-69f2f6e236f2 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:16.394132 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 569/1146] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:25:16 2022] GET /identity/v3/role_assignments?user.id=a0f66dbcf967418594605d16a971f253&scope.project.id=16fb5037512748fa825f494a820ae384&role.id=c1e514a4b5424f959b069f36cbfd8147 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:25:18.350730 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ff593-4475-4099-b9a8-2ad7fccc0797 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:18.351275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ff593-4475-4099-b9a8-2ad7fccc0797 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:18.351275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ff593-4475-4099-b9a8-2ad7fccc0797 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:18.352604 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 578/1147] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:25:18.357789 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0078bff-16ee-4db7-911e-43c12e168255 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:18.358682 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0078bff-16ee-4db7-911e-43c12e168255 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:18.358825 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0078bff-16ee-4db7-911e-43c12e168255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:18.377555 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-e0078bff-16ee-4db7-911e-43c12e168255 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:25:18.378714 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e0078bff-16ee-4db7-911e-43c12e168255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:25:18.442891 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 570/1148] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:25:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:25:18.464390 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97ff593-4475-4099-b9a8-2ad7fccc0797 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:18.514720 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:18.563464 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97ff593-4475-4099-b9a8-2ad7fccc0797 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:18.595429 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae938d19-8f52-43e6-b3c9-9d76c2b13add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:18.596043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae938d19-8f52-43e6-b3c9-9d76c2b13add None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:18.596383 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae938d19-8f52-43e6-b3c9-9d76c2b13add None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:18.596593 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae938d19-8f52-43e6-b3c9-9d76c2b13add None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:18.606303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae938d19-8f52-43e6-b3c9-9d76c2b13add None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:25:18.611509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-ae938d19-8f52-43e6-b3c9-9d76c2b13add None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:25:18.611509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:25:18.611509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:25:18.611509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:25:18.611509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:25:18.611509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:25:18.611509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:25:18.611509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:25:18.611509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:25:18.611509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:25:18.611509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:25:18.611509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:25:18.611509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:25:18.611509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:25:18.611509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:25:18.611509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:25:18.612589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:25:18.612589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:25:18.612589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:25:18.612589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:25:18.612589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:25:18.612589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:25:18.612589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:25:18.612589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:25:18.612589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:25:18.612589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:25:18.612589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:25:18.612589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:25:18.612589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:25:18.612589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:25:18.612589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:25:18.612589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:25:18.613864 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:25:18.613864 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:25:18.613864 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:25:18.613864 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:25:18.613864 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:25:18.613864 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:25:18.613864 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:25:18.613864 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:25:18.613864 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:25:18.613864 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:25:18.613864 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:25:18.613864 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:25:18.613864 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 579/1149] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:25:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:25:18.618537 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0078bff-16ee-4db7-911e-43c12e168255 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:18.681320 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:18.713606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0078bff-16ee-4db7-911e-43c12e168255 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:18.745976 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00dffe4-4919-4891-9706-d0e1ade3cdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:18.746570 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00dffe4-4919-4891-9706-d0e1ade3cdcb None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:18.746848 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00dffe4-4919-4891-9706-d0e1ade3cdcb None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:18.747118 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00dffe4-4919-4891-9706-d0e1ade3cdcb None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:18.747775 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00dffe4-4919-4891-9706-d0e1ade3cdcb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:25:18.748075 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00dffe4-4919-4891-9706-d0e1ade3cdcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:18.750060 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00dffe4-4919-4891-9706-d0e1ade3cdcb None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:18.757730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 571/1150] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:25:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:18.763572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae938d19-8f52-43e6-b3c9-9d76c2b13add None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:18.860402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae938d19-8f52-43e6-b3c9-9d76c2b13add None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:18.870445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:18.893471 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8783a18-2513-4cba-8061-f04f179c6410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:18.894089 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8783a18-2513-4cba-8061-f04f179c6410 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:18.894365 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8783a18-2513-4cba-8061-f04f179c6410 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:18.894639 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8783a18-2513-4cba-8061-f04f179c6410 None admin] PATH_INFO: `/v3/users/a0f66dbcf967418594605d16a971f253` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:18.895788 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8783a18-2513-4cba-8061-f04f179c6410 None admin] RBAC: Authorizing `identity:get_user(user_id=a0f66dbcf967418594605d16a971f253)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:18.897247 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8783a18-2513-4cba-8061-f04f179c6410 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:18.898793 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 580/1151] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:25:18 2022] GET /identity/v3/users/a0f66dbcf967418594605d16a971f253 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:18.904638 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00dffe4-4919-4891-9706-d0e1ade3cdcb None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:19.002109 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00dffe4-4919-4891-9706-d0e1ade3cdcb None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:19.027112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:19.034726 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff22e52-329a-46d6-bb69-1e40421abb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:19.038180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff22e52-329a-46d6-bb69-1e40421abb8e None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:19.038180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff22e52-329a-46d6-bb69-1e40421abb8e None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:19.038180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff22e52-329a-46d6-bb69-1e40421abb8e None admin] PATH_INFO: `/v3/projects/16fb5037512748fa825f494a820ae384` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:19.038180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff22e52-329a-46d6-bb69-1e40421abb8e None admin] RBAC: Authorizing `identity:get_project(project_id=16fb5037512748fa825f494a820ae384)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:19.038499 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff22e52-329a-46d6-bb69-1e40421abb8e None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:19.040065 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 572/1152] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:25:18 2022] GET /identity/v3/projects/16fb5037512748fa825f494a820ae384 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:19.046402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8783a18-2513-4cba-8061-f04f179c6410 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:19.142980 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8783a18-2513-4cba-8061-f04f179c6410 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:19.175274 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e82d7-fa6d-4193-bcac-96e2e8ab277f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:19.175912 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e82d7-fa6d-4193-bcac-96e2e8ab277f None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:19.176187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e82d7-fa6d-4193-bcac-96e2e8ab277f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:19.176454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e82d7-fa6d-4193-bcac-96e2e8ab277f None admin] PATH_INFO: `/v3/projects/16fb5037512748fa825f494a820ae384/users/a0f66dbcf967418594605d16a971f253/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:19.178312 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e82d7-fa6d-4193-bcac-96e2e8ab277f None admin] RBAC: Authorizing `identity:create_grant(project_id=16fb5037512748fa825f494a820ae384, user_id=a0f66dbcf967418594605d16a971f253, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:19.179728 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e82d7-fa6d-4193-bcac-96e2e8ab277f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:19.286014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 581/1153] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 08:25:19 2022] PUT /identity/v3/projects/16fb5037512748fa825f494a820ae384/users/a0f66dbcf967418594605d16a971f253/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:25:21.247168 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c8157-070f-490b-836c-b7ddf858fa78 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:21.247702 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c8157-070f-490b-836c-b7ddf858fa78 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:21.247702 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c8157-070f-490b-836c-b7ddf858fa78 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:21.249662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 573/1154] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:25:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:25:21.254773 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28cd572-3059-460a-8e10-6baaaba5254f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:21.255148 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28cd572-3059-460a-8e10-6baaaba5254f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:21.255318 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28cd572-3059-460a-8e10-6baaaba5254f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:21.273959 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c28cd572-3059-460a-8e10-6baaaba5254f None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:25:21.275121 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c28cd572-3059-460a-8e10-6baaaba5254f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:25:21.302795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:21.338793 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 582/1155] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:25:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:25:21.362145 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c8157-070f-490b-836c-b7ddf858fa78 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:21.463359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c8157-070f-490b-836c-b7ddf858fa78 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:21.495112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629da47-1292-4d90-9470-be3c214ae8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:21.495727 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629da47-1292-4d90-9470-be3c214ae8f5 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:21.496002 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629da47-1292-4d90-9470-be3c214ae8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:21.496268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629da47-1292-4d90-9470-be3c214ae8f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:21.507126 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f629da47-1292-4d90-9470-be3c214ae8f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:25:21.511657 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-f629da47-1292-4d90-9470-be3c214ae8f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:25:21.511657 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:25:21.511657 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:25:21.511657 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:25:21.511657 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:25:21.511657 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:25:21.511657 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:25:21.511657 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:25:21.511657 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:25:21.511657 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:25:21.511657 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:25:21.511657 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:25:21.511657 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:25:21.511657 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:25:21.511657 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:25:21.511657 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:25:21.512729 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:25:21.512729 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:25:21.512729 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:25:21.512729 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:25:21.512729 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:25:21.512729 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:25:21.512729 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:25:21.512729 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:25:21.512729 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:25:21.512729 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:25:21.512729 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:25:21.512729 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:25:21.512729 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:25:21.512729 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:25:21.512729 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:25:21.512729 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:25:21.513725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:25:21.513725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:25:21.513725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:25:21.513725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:25:21.513725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:25:21.513725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:25:21.513725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:25:21.513725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:25:21.513725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:25:21.513725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:25:21.513725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:25:21.513725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:25:21.513725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 574/1156] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:25:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:25:21.518932 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28cd572-3059-460a-8e10-6baaaba5254f None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:21.613401 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28cd572-3059-460a-8e10-6baaaba5254f None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:21.646073 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2bbe7-8ceb-4b29-a03e-86f765d6d2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:21.646663 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2bbe7-8ceb-4b29-a03e-86f765d6d2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:21.646939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2bbe7-8ceb-4b29-a03e-86f765d6d2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:21.647206 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2bbe7-8ceb-4b29-a03e-86f765d6d2c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:21.647854 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac2bbe7-8ceb-4b29-a03e-86f765d6d2c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:25:21.648148 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac2bbe7-8ceb-4b29-a03e-86f765d6d2c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:21.649526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac2bbe7-8ceb-4b29-a03e-86f765d6d2c7 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:21.657260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 583/1157] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:25:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:21.662855 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629da47-1292-4d90-9470-be3c214ae8f5 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:21.717860 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:21.759747 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629da47-1292-4d90-9470-be3c214ae8f5 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:21.791958 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64de5d-15e4-4a81-86b2-86ef9f607c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:21.792568 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64de5d-15e4-4a81-86b2-86ef9f607c71 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:21.792842 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64de5d-15e4-4a81-86b2-86ef9f607c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:21.793108 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64de5d-15e4-4a81-86b2-86ef9f607c71 None admin] PATH_INFO: `/v3/users/a0f66dbcf967418594605d16a971f253` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:21.794220 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b64de5d-15e4-4a81-86b2-86ef9f607c71 None admin] RBAC: Authorizing `identity:get_user(user_id=a0f66dbcf967418594605d16a971f253)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:21.795732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b64de5d-15e4-4a81-86b2-86ef9f607c71 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:21.797310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 575/1158] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:25:21 2022] GET /identity/v3/users/a0f66dbcf967418594605d16a971f253 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:25:21.802929 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2bbe7-8ceb-4b29-a03e-86f765d6d2c7 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:21.832786 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:21.899714 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2bbe7-8ceb-4b29-a03e-86f765d6d2c7 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:21.931116 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544dc30f-1e10-4d73-8dda-0ed8e99913f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:21.931704 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544dc30f-1e10-4d73-8dda-0ed8e99913f0 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:21.931979 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544dc30f-1e10-4d73-8dda-0ed8e99913f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:21.932249 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544dc30f-1e10-4d73-8dda-0ed8e99913f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:21.996613 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544dc30f-1e10-4d73-8dda-0ed8e99913f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:21.998079 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544dc30f-1e10-4d73-8dda-0ed8e99913f0 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:22.026445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:22.093828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 584/1159] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:25:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 08:25:22.099653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64de5d-15e4-4a81-86b2-86ef9f607c71 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:22.199854 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64de5d-15e4-4a81-86b2-86ef9f607c71 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:22.221969 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:22.232177 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc7ff4-c9f9-4b6f-9be6-bd773899dc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:22.232795 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc7ff4-c9f9-4b6f-9be6-bd773899dc92 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:22.233613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc7ff4-c9f9-4b6f-9be6-bd773899dc92 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:22.233952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc7ff4-c9f9-4b6f-9be6-bd773899dc92 None admin] PATH_INFO: `/v3/projects/16fb5037512748fa825f494a820ae384` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:22.234986 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc7ff4-c9f9-4b6f-9be6-bd773899dc92 None admin] RBAC: Authorizing `identity:get_project(project_id=16fb5037512748fa825f494a820ae384)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:22.236413 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc7ff4-c9f9-4b6f-9be6-bd773899dc92 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:22.238045 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 576/1160] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:25:22 2022] GET /identity/v3/projects/16fb5037512748fa825f494a820ae384 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:22.244156 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544dc30f-1e10-4d73-8dda-0ed8e99913f0 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:22.340433 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544dc30f-1e10-4d73-8dda-0ed8e99913f0 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:22.368047 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:22.373660 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39367e4-0245-48b9-bc90-581481cb1889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:22.374260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39367e4-0245-48b9-bc90-581481cb1889 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:22.374542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39367e4-0245-48b9-bc90-581481cb1889 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:22.374817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39367e4-0245-48b9-bc90-581481cb1889 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:22.375535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39367e4-0245-48b9-bc90-581481cb1889 None admin] RBAC: Adding query filter params (role.id=c1e514a4b5424f959b069f36cbfd8147, scope.project.id=16fb5037512748fa825f494a820ae384, user.id=a0f66dbcf967418594605d16a971f253) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:25:22.375842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39367e4-0245-48b9-bc90-581481cb1889 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:22.377293 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39367e4-0245-48b9-bc90-581481cb1889 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:22.382948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 585/1161] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:25:22 2022] GET /identity/v3/role_assignments?user.id=a0f66dbcf967418594605d16a971f253&scope.project.id=16fb5037512748fa825f494a820ae384&role.id=c1e514a4b5424f959b069f36cbfd8147 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:24.400832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcd36a-c41a-47bc-8c8f-0d89577e4b62 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:24.401417 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcd36a-c41a-47bc-8c8f-0d89577e4b62 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:24.401417 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcd36a-c41a-47bc-8c8f-0d89577e4b62 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:24.402753 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 577/1162] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:25:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:25:24.408061 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07365533-463b-419a-a0d5-98d2bcf654fc None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:24.408445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07365533-463b-419a-a0d5-98d2bcf654fc None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:24.408598 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07365533-463b-419a-a0d5-98d2bcf654fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:24.427361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-07365533-463b-419a-a0d5-98d2bcf654fc None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:25:24.428535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-07365533-463b-419a-a0d5-98d2bcf654fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:25:24.493891 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 586/1163] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:25:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:25:24.516276 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bcd36a-c41a-47bc-8c8f-0d89577e4b62 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:24.617422 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bcd36a-c41a-47bc-8c8f-0d89577e4b62 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:24.649548 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98314f45-2cc2-4990-b479-e82241744a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:24.650137 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98314f45-2cc2-4990-b479-e82241744a68 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:24.650415 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98314f45-2cc2-4990-b479-e82241744a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:24.650684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98314f45-2cc2-4990-b479-e82241744a68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:24.706396 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:24.715368 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98314f45-2cc2-4990-b479-e82241744a68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:24.716785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98314f45-2cc2-4990-b479-e82241744a68 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:24.810202 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 578/1164] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:25:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 08:25:24.816623 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07365533-463b-419a-a0d5-98d2bcf654fc None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:24.911314 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07365533-463b-419a-a0d5-98d2bcf654fc None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:24.933437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:24.944526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd438e75-4185-43b5-870b-124bb645bf00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:24.956198 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd438e75-4185-43b5-870b-124bb645bf00 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:24.956198 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd438e75-4185-43b5-870b-124bb645bf00 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:24.956198 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd438e75-4185-43b5-870b-124bb645bf00 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:24.956198 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd438e75-4185-43b5-870b-124bb645bf00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:24.956198 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd438e75-4185-43b5-870b-124bb645bf00 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:24.965511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-dd438e75-4185-43b5-870b-124bb645bf00 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 08 08:25:24.965511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:25:24.965511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 08 08:25:24.965511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 08 08:25:24.965511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:25:24.965511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:25:24.965511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:25:24.965511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:25:24.965511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 08 08:25:24.965511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:25:24.965511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:25:24.965511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:25:24.965511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:25:24.965511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:25:24.965511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:25:24.965511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:25:24.965511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:25:24.966954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:25:24.966954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:25:24.966954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:25:24.966954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:25:24.966954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:25:24.966954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:25:24.966954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:25:24.966954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:25:24.966954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:25:24.966954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:25:24.966954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:25:24.966954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:25:24.966954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:25:24.966954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:25:24.966954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:25:24.966954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:25:24.968181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:25:24.968181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:25:24.968181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:25:24.968181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:25:24.968181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:25:24.968181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:25:24.968181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:25:24.968181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:25:24.968181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:25:24.968181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:25:24.968181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 08 08:25:24.968181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:25:24.968181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 08 08:25:24.968181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:25:24.968181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 587/1165] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Aug 8 08:25:24 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:25:24.972726 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98314f45-2cc2-4990-b479-e82241744a68 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:25.058195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:25.066524 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98314f45-2cc2-4990-b479-e82241744a68 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:25.099450 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422bdcfc-9dda-4dbd-885f-329d39057f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:25.100035 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422bdcfc-9dda-4dbd-885f-329d39057f92 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:25.100309 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422bdcfc-9dda-4dbd-885f-329d39057f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:25.100576 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422bdcfc-9dda-4dbd-885f-329d39057f92 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:25.101222 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422bdcfc-9dda-4dbd-885f-329d39057f92 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:25:25.101518 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422bdcfc-9dda-4dbd-885f-329d39057f92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:25.102883 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422bdcfc-9dda-4dbd-885f-329d39057f92 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:25.110594 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 579/1166] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Aug 8 08:25:24 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:27.088434 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a8cd5-3507-4859-900f-0d3df23d5740 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:27.088959 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a8cd5-3507-4859-900f-0d3df23d5740 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:27.088959 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a8cd5-3507-4859-900f-0d3df23d5740 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:27.090343 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 588/1167] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:25:27.096131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59949e26-9dc3-4614-ad9c-c613b432b635 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:27.096512 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59949e26-9dc3-4614-ad9c-c613b432b635 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:27.096682 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59949e26-9dc3-4614-ad9c-c613b432b635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:27.115577 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-59949e26-9dc3-4614-ad9c-c613b432b635 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:25:27.116741 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-59949e26-9dc3-4614-ad9c-c613b432b635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:25:27.180840 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 580/1168] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:25:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:25:27.203032 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a8cd5-3507-4859-900f-0d3df23d5740 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:27.302589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a8cd5-3507-4859-900f-0d3df23d5740 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:27.334271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ff9b4-c783-4fcc-a4f1-104abd21452e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:27.334863 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ff9b4-c783-4fcc-a4f1-104abd21452e None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:27.335145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ff9b4-c783-4fcc-a4f1-104abd21452e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:27.335420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ff9b4-c783-4fcc-a4f1-104abd21452e None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:27.336320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822ff9b4-c783-4fcc-a4f1-104abd21452e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:27.337718 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822ff9b4-c783-4fcc-a4f1-104abd21452e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:27.418864 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 589/1169] 10.222.0.33 () {60 vars in 1138 bytes} [Mon Aug 8 08:25:27 2022] POST /identity/v3/domains => generated 267 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:25:29.439872 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96240f-2438-4029-86bb-55048ada4642 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:29.440285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96240f-2438-4029-86bb-55048ada4642 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:29.440452 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96240f-2438-4029-86bb-55048ada4642 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:29.441833 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 581/1170] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:25:29.446879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3de1c3-aafd-4984-bbf5-e2fa85279379 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:29.447254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3de1c3-aafd-4984-bbf5-e2fa85279379 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:29.447439 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3de1c3-aafd-4984-bbf5-e2fa85279379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:29.465485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ad3de1c3-aafd-4984-bbf5-e2fa85279379 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:25:29.466655 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ad3de1c3-aafd-4984-bbf5-e2fa85279379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:25:29.505416 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:29.531553 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 590/1171] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:25:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:25:29.558694 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96240f-2438-4029-86bb-55048ada4642 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:29.642401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:29.657209 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96240f-2438-4029-86bb-55048ada4642 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:29.690278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6255e1-c726-4dec-83fd-8dbcf6a631c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:29.690834 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6255e1-c726-4dec-83fd-8dbcf6a631c5 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:29.691113 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6255e1-c726-4dec-83fd-8dbcf6a631c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:29.691386 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6255e1-c726-4dec-83fd-8dbcf6a631c5 None admin] PATH_INFO: `/v3/domains/1a39cc8510904026a0262ffbdb576394` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:29.701277 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6255e1-c726-4dec-83fd-8dbcf6a631c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1a39cc8510904026a0262ffbdb576394)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:29.702751 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6255e1-c726-4dec-83fd-8dbcf6a631c5 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:29.704305 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 582/1172] 10.222.0.33 () {56 vars in 1183 bytes} [Mon Aug 8 08:25:29 2022] GET /identity/v3/domains/1a39cc8510904026a0262ffbdb576394 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:29.710182 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3de1c3-aafd-4984-bbf5-e2fa85279379 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:29.804841 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3de1c3-aafd-4984-bbf5-e2fa85279379 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:29.837470 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a9972-f84c-4d6b-9d4a-2c3f5470e60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:29.838063 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a9972-f84c-4d6b-9d4a-2c3f5470e60d None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:29.838337 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a9972-f84c-4d6b-9d4a-2c3f5470e60d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:29.838604 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a9972-f84c-4d6b-9d4a-2c3f5470e60d None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:29.839239 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a9972-f84c-4d6b-9d4a-2c3f5470e60d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:29.840658 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a9972-f84c-4d6b-9d4a-2c3f5470e60d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:29.898663 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 591/1173] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Aug 8 08:25:29 2022] POST /identity/v3/projects => generated 361 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:25:31.908833 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf3200-d15c-415a-b124-fd4ed0c911d6 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:31.909408 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf3200-d15c-415a-b124-fd4ed0c911d6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:31.909512 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf3200-d15c-415a-b124-fd4ed0c911d6 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:31.910783 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 583/1174] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:25:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:25:31.915790 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e3a83-f410-42a7-879b-95516e1e60ed None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:31.916204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e3a83-f410-42a7-879b-95516e1e60ed None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:31.916332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e3a83-f410-42a7-879b-95516e1e60ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:31.934011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-286e3a83-f410-42a7-879b-95516e1e60ed None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:25:31.935180 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-286e3a83-f410-42a7-879b-95516e1e60ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:25:31.955231 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:32.000094 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 592/1175] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:25:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:25:32.024514 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf3200-d15c-415a-b124-fd4ed0c911d6 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:32.122748 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf3200-d15c-415a-b124-fd4ed0c911d6 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:32.154507 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c07723-38ea-4e02-8c4d-77804e495aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:32.156045 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c07723-38ea-4e02-8c4d-77804e495aeb None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:32.156320 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c07723-38ea-4e02-8c4d-77804e495aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:32.156588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c07723-38ea-4e02-8c4d-77804e495aeb None admin] PATH_INFO: `/v3/domains/1a39cc8510904026a0262ffbdb576394` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:32.157699 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c07723-38ea-4e02-8c4d-77804e495aeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=1a39cc8510904026a0262ffbdb576394)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:32.159173 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c07723-38ea-4e02-8c4d-77804e495aeb None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:32.160656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 584/1176] 10.222.0.33 () {56 vars in 1183 bytes} [Mon Aug 8 08:25:32 2022] GET /identity/v3/domains/1a39cc8510904026a0262ffbdb576394 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:32.166413 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e3a83-f410-42a7-879b-95516e1e60ed None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:32.263444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e3a83-f410-42a7-879b-95516e1e60ed None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:32.295981 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc1a037-b2d2-4402-bb30-3e9ef53b6d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:32.296565 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1a037-b2d2-4402-bb30-3e9ef53b6d38 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:32.296843 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1a037-b2d2-4402-bb30-3e9ef53b6d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:32.297112 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1a037-b2d2-4402-bb30-3e9ef53b6d38 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:32.297783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc1a037-b2d2-4402-bb30-3e9ef53b6d38 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:32.299204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc1a037-b2d2-4402-bb30-3e9ef53b6d38 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:32.371699 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 593/1177] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 08:25:32 2022] POST /identity/v3/users => generated 311 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:25:34.388112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a6e4f-914a-47ea-b8d5-e16b28787bdf None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:34.388643 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a6e4f-914a-47ea-b8d5-e16b28787bdf None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:34.388643 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a6e4f-914a-47ea-b8d5-e16b28787bdf None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:34.390608 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 585/1178] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:25:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:25:34.395637 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18287c-6bdb-4c2d-b675-2b20fe6c2bdc None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:34.396046 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18287c-6bdb-4c2d-b675-2b20fe6c2bdc None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:34.396183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18287c-6bdb-4c2d-b675-2b20fe6c2bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:34.414540 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-3b18287c-6bdb-4c2d-b675-2b20fe6c2bdc None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:25:34.415761 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3b18287c-6bdb-4c2d-b675-2b20fe6c2bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:25:34.423148 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:34.480264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 594/1179] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:25:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:25:34.503168 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a6e4f-914a-47ea-b8d5-e16b28787bdf None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:34.520284 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:34.608895 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a6e4f-914a-47ea-b8d5-e16b28787bdf None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:34.641330 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a958280e-5791-4f1e-a95e-b97e40aa8958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:34.641884 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a958280e-5791-4f1e-a95e-b97e40aa8958 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:34.642159 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a958280e-5791-4f1e-a95e-b97e40aa8958 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:34.642422 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a958280e-5791-4f1e-a95e-b97e40aa8958 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:34.652477 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a958280e-5791-4f1e-a95e-b97e40aa8958 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:25:34.657326 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-a958280e-5791-4f1e-a95e-b97e40aa8958 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:25:34.657326 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:25:34.657326 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:25:34.657326 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:25:34.657326 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:25:34.657326 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:25:34.657326 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:25:34.657326 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:25:34.657326 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:25:34.657326 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:25:34.657326 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:25:34.657326 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:25:34.657326 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:25:34.657326 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:25:34.657326 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:25:34.657326 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:25:34.658409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:25:34.658409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:25:34.658409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:25:34.658409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:25:34.658409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:25:34.658409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:25:34.658409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:25:34.658409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:25:34.658409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:25:34.658409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:25:34.658409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:25:34.658409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:25:34.658409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:25:34.658409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:25:34.658409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:25:34.658409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:25:34.659296 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:25:34.659296 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:25:34.659296 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:25:34.659296 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:25:34.659296 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:25:34.659296 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:25:34.659296 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:25:34.659296 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:25:34.659296 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:25:34.659296 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:25:34.659296 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:25:34.659296 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:25:34.659296 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 586/1180] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:25:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:25:34.664464 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18287c-6bdb-4c2d-b675-2b20fe6c2bdc None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:34.759130 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18287c-6bdb-4c2d-b675-2b20fe6c2bdc None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:34.791652 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532e2d9-5ef9-4929-b609-b3ec8235edcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:34.792244 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532e2d9-5ef9-4929-b609-b3ec8235edcc None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:34.792518 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532e2d9-5ef9-4929-b609-b3ec8235edcc None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:34.792786 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532e2d9-5ef9-4929-b609-b3ec8235edcc None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:34.793471 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8532e2d9-5ef9-4929-b609-b3ec8235edcc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:25:34.793770 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8532e2d9-5ef9-4929-b609-b3ec8235edcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:34.795127 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8532e2d9-5ef9-4929-b609-b3ec8235edcc None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:34.802899 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 595/1181] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:25:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:34.808571 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a958280e-5791-4f1e-a95e-b97e40aa8958 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:34.881019 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:34.904438 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a958280e-5791-4f1e-a95e-b97e40aa8958 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:34.937837 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f1dcc-954c-4ce4-ab03-e366b2ebf4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:34.938458 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f1dcc-954c-4ce4-ab03-e366b2ebf4fc None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:34.938738 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f1dcc-954c-4ce4-ab03-e366b2ebf4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:34.939006 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f1dcc-954c-4ce4-ab03-e366b2ebf4fc None admin] PATH_INFO: `/v3/users/dc27c428933a4e34bdd3fedfba9f507f` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:34.951624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22f1dcc-954c-4ce4-ab03-e366b2ebf4fc None admin] RBAC: Authorizing `identity:get_user(user_id=dc27c428933a4e34bdd3fedfba9f507f)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:34.953202 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22f1dcc-954c-4ce4-ab03-e366b2ebf4fc None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:34.954815 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 587/1182] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:25:34 2022] GET /identity/v3/users/dc27c428933a4e34bdd3fedfba9f507f => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:34.960634 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532e2d9-5ef9-4929-b609-b3ec8235edcc None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:34.964628 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:35.057395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532e2d9-5ef9-4929-b609-b3ec8235edcc None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:35.089713 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239deddf-41e4-4ce2-ba92-2c9d06c98f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:35.090302 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239deddf-41e4-4ce2-ba92-2c9d06c98f0e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:35.090577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239deddf-41e4-4ce2-ba92-2c9d06c98f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:35.090842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239deddf-41e4-4ce2-ba92-2c9d06c98f0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:35.154424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239deddf-41e4-4ce2-ba92-2c9d06c98f0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:35.156331 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239deddf-41e4-4ce2-ba92-2c9d06c98f0e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:35.158775 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:35.250275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 596/1183] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:25:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 08:25:35.256124 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f1dcc-954c-4ce4-ab03-e366b2ebf4fc None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:35.350432 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f1dcc-954c-4ce4-ab03-e366b2ebf4fc None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:35.378092 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:35.382702 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3654be0-e2b4-4a0e-b2d7-43f25c79481a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:35.383314 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3654be0-e2b4-4a0e-b2d7-43f25c79481a None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:35.383588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3654be0-e2b4-4a0e-b2d7-43f25c79481a None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:35.383853 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3654be0-e2b4-4a0e-b2d7-43f25c79481a None admin] PATH_INFO: `/v3/projects/99e686a7984c450dbd26cf1f8399d86a` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:35.384928 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3654be0-e2b4-4a0e-b2d7-43f25c79481a None admin] RBAC: Authorizing `identity:get_project(project_id=99e686a7984c450dbd26cf1f8399d86a)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:35.386359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3654be0-e2b4-4a0e-b2d7-43f25c79481a None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:35.387953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 588/1184] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:25:35 2022] GET /identity/v3/projects/99e686a7984c450dbd26cf1f8399d86a => generated 361 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:35.394680 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239deddf-41e4-4ce2-ba92-2c9d06c98f0e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:35.490225 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239deddf-41e4-4ce2-ba92-2c9d06c98f0e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:35.492464 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:35.523990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21511e-0e37-4ba2-939d-201a9bb1c80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:35.524582 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21511e-0e37-4ba2-939d-201a9bb1c80c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:35.524858 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21511e-0e37-4ba2-939d-201a9bb1c80c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:35.525127 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21511e-0e37-4ba2-939d-201a9bb1c80c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:35.525869 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21511e-0e37-4ba2-939d-201a9bb1c80c None admin] RBAC: Adding query filter params (role.id=c1e514a4b5424f959b069f36cbfd8147, scope.project.id=99e686a7984c450dbd26cf1f8399d86a, user.id=dc27c428933a4e34bdd3fedfba9f507f) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:25:35.526167 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21511e-0e37-4ba2-939d-201a9bb1c80c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:35.527581 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21511e-0e37-4ba2-939d-201a9bb1c80c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:35.532730 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 597/1185] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:25:35 2022] GET /identity/v3/role_assignments?user.id=dc27c428933a4e34bdd3fedfba9f507f&scope.project.id=99e686a7984c450dbd26cf1f8399d86a&role.id=c1e514a4b5424f959b069f36cbfd8147 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:37.475744 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfccc90-21b9-449f-a3c0-d19f1ff9672b None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:37.476241 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfccc90-21b9-449f-a3c0-d19f1ff9672b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:37.476341 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfccc90-21b9-449f-a3c0-d19f1ff9672b None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:37.477663 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 589/1186] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:25:37.482571 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b0dd2-a683-47cc-8e4e-11e8668d3644 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:37.493087 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b0dd2-a683-47cc-8e4e-11e8668d3644 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:37.493087 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b0dd2-a683-47cc-8e4e-11e8668d3644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:37.501067 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-854b0dd2-a683-47cc-8e4e-11e8668d3644 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:25:37.502241 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-854b0dd2-a683-47cc-8e4e-11e8668d3644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:25:37.564962 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 598/1187] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:25:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:25:37.587784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfccc90-21b9-449f-a3c0-d19f1ff9672b None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:37.685825 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfccc90-21b9-449f-a3c0-d19f1ff9672b None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:37.718968 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fae886-0b84-45de-87f1-e62ef538b9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:37.719572 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fae886-0b84-45de-87f1-e62ef538b9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:37.719846 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fae886-0b84-45de-87f1-e62ef538b9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:37.720108 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fae886-0b84-45de-87f1-e62ef538b9a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:37.729723 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fae886-0b84-45de-87f1-e62ef538b9a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:25:37.734301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-98fae886-0b84-45de-87f1-e62ef538b9a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:25:37.734301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:25:37.734301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:25:37.734301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:25:37.734301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:25:37.734301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:25:37.734301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:25:37.734301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:25:37.734301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:25:37.734301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:25:37.734301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:25:37.734301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:25:37.734301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:25:37.734301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:25:37.734301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:25:37.734301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:25:37.735411 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:25:37.735411 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:25:37.735411 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:25:37.735411 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:25:37.735411 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:25:37.735411 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:25:37.735411 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:25:37.735411 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:25:37.735411 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:25:37.735411 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:25:37.735411 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:25:37.735411 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:25:37.735411 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:25:37.735411 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:25:37.735411 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:25:37.735411 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:25:37.736281 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:25:37.736281 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:25:37.736281 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:25:37.736281 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:25:37.736281 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:25:37.736281 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:25:37.736281 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:25:37.736281 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:25:37.736281 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:25:37.736281 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:25:37.736281 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:25:37.736281 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:25:37.736281 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 590/1188] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:25:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:25:37.741264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854b0dd2-a683-47cc-8e4e-11e8668d3644 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:37.832703 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:37.836085 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854b0dd2-a683-47cc-8e4e-11e8668d3644 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:37.868669 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89279c4d-119f-48b3-ab5e-b09a6a1c8d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:37.869285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89279c4d-119f-48b3-ab5e-b09a6a1c8d11 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:37.869564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89279c4d-119f-48b3-ab5e-b09a6a1c8d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:37.869831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89279c4d-119f-48b3-ab5e-b09a6a1c8d11 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:37.870481 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89279c4d-119f-48b3-ab5e-b09a6a1c8d11 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:25:37.870777 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89279c4d-119f-48b3-ab5e-b09a6a1c8d11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:37.872130 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89279c4d-119f-48b3-ab5e-b09a6a1c8d11 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:37.879967 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 599/1189] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:25:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:37.885758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fae886-0b84-45de-87f1-e62ef538b9a9 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:37.944356 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:37.983775 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fae886-0b84-45de-87f1-e62ef538b9a9 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:38.016259 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f96eda-3278-4c7d-93ec-64b41ad1abd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:38.016870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f96eda-3278-4c7d-93ec-64b41ad1abd2 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:38.017174 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f96eda-3278-4c7d-93ec-64b41ad1abd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:38.017454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f96eda-3278-4c7d-93ec-64b41ad1abd2 None admin] PATH_INFO: `/v3/users/dc27c428933a4e34bdd3fedfba9f507f` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:38.018558 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f96eda-3278-4c7d-93ec-64b41ad1abd2 None admin] RBAC: Authorizing `identity:get_user(user_id=dc27c428933a4e34bdd3fedfba9f507f)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:38.019986 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f96eda-3278-4c7d-93ec-64b41ad1abd2 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:38.021579 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 591/1190] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:25:37 2022] GET /identity/v3/users/dc27c428933a4e34bdd3fedfba9f507f => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:38.027155 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89279c4d-119f-48b3-ab5e-b09a6a1c8d11 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:38.124532 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89279c4d-119f-48b3-ab5e-b09a6a1c8d11 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:38.156913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08fb3d-1ef1-493a-8b26-19da978cea3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:38.157549 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08fb3d-1ef1-493a-8b26-19da978cea3e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:38.157829 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08fb3d-1ef1-493a-8b26-19da978cea3e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:38.158100 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08fb3d-1ef1-493a-8b26-19da978cea3e None admin] PATH_INFO: `/v3/projects/99e686a7984c450dbd26cf1f8399d86a` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:38.159201 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b08fb3d-1ef1-493a-8b26-19da978cea3e None admin] RBAC: Authorizing `identity:get_project(project_id=99e686a7984c450dbd26cf1f8399d86a)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:38.160642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b08fb3d-1ef1-493a-8b26-19da978cea3e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:38.162275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 600/1191] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:25:38 2022] GET /identity/v3/projects/99e686a7984c450dbd26cf1f8399d86a => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:38.168209 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f96eda-3278-4c7d-93ec-64b41ad1abd2 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:38.265051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f96eda-3278-4c7d-93ec-64b41ad1abd2 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:38.290237 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:38.297730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a2e32-2ec5-4ed3-b85b-4a2ffd8b91bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:38.299832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a2e32-2ec5-4ed3-b85b-4a2ffd8b91bb None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:38.299832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a2e32-2ec5-4ed3-b85b-4a2ffd8b91bb None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:38.299832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a2e32-2ec5-4ed3-b85b-4a2ffd8b91bb None admin] PATH_INFO: `/v3/projects/99e686a7984c450dbd26cf1f8399d86a/users/dc27c428933a4e34bdd3fedfba9f507f/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:38.300750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0a2e32-2ec5-4ed3-b85b-4a2ffd8b91bb None admin] RBAC: Authorizing `identity:create_grant(project_id=99e686a7984c450dbd26cf1f8399d86a, user_id=dc27c428933a4e34bdd3fedfba9f507f, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:38.302184 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0a2e32-2ec5-4ed3-b85b-4a2ffd8b91bb None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:38.361056 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 592/1192] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 08:25:38 2022] PUT /identity/v3/projects/99e686a7984c450dbd26cf1f8399d86a/users/dc27c428933a4e34bdd3fedfba9f507f/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:25:40.315244 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69c2ff-5732-4d82-b566-d1e9d1aff3c7 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:40.315780 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69c2ff-5732-4d82-b566-d1e9d1aff3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:40.315780 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69c2ff-5732-4d82-b566-d1e9d1aff3c7 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:40.317120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 601/1193] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:25:40.322281 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d6b42-f90a-46d2-887c-c035283bdbe5 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:40.322678 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d6b42-f90a-46d2-887c-c035283bdbe5 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:40.322823 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d6b42-f90a-46d2-887c-c035283bdbe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:40.341037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-c87d6b42-f90a-46d2-887c-c035283bdbe5 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:25:40.342228 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c87d6b42-f90a-46d2-887c-c035283bdbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:25:40.406648 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 593/1194] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:25:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:25:40.429816 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69c2ff-5732-4d82-b566-d1e9d1aff3c7 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:40.458384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:40.529634 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69c2ff-5732-4d82-b566-d1e9d1aff3c7 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:40.563090 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c364dd-98c2-49e0-980e-3c43da8ac036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:40.563710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c364dd-98c2-49e0-980e-3c43da8ac036 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:40.563998 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c364dd-98c2-49e0-980e-3c43da8ac036 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:40.564259 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c364dd-98c2-49e0-980e-3c43da8ac036 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:40.574417 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c364dd-98c2-49e0-980e-3c43da8ac036 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:25:40.579113 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-12c364dd-98c2-49e0-980e-3c43da8ac036 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:25:40.579113 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:25:40.579113 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:25:40.579113 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:25:40.579113 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:25:40.579113 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:25:40.579113 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:25:40.579113 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:25:40.579113 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:25:40.579113 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:25:40.579113 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:25:40.579113 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:25:40.579113 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:25:40.579113 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:25:40.579113 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:25:40.579113 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:25:40.580204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:25:40.580204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:25:40.580204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:25:40.580204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:25:40.580204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:25:40.580204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:25:40.580204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:25:40.580204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:25:40.580204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:25:40.580204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:25:40.580204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:25:40.580204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:25:40.580204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:25:40.580204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:25:40.580204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:25:40.580204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:25:40.581434 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:25:40.581434 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:25:40.581434 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:25:40.581434 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:25:40.581434 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:25:40.581434 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:25:40.581434 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:25:40.581434 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:25:40.581434 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:25:40.581434 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:25:40.581434 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:25:40.581434 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:25:40.581434 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 602/1195] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:25:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:25:40.586253 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d6b42-f90a-46d2-887c-c035283bdbe5 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:40.681159 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d6b42-f90a-46d2-887c-c035283bdbe5 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:40.697361 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:40.713570 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d421a-660f-4402-9a78-81366f0efe76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:40.714248 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d421a-660f-4402-9a78-81366f0efe76 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:40.714523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d421a-660f-4402-9a78-81366f0efe76 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:40.714787 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d421a-660f-4402-9a78-81366f0efe76 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:40.715437 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d421a-660f-4402-9a78-81366f0efe76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:25:40.715731 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d421a-660f-4402-9a78-81366f0efe76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:40.717066 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d421a-660f-4402-9a78-81366f0efe76 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:40.729185 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 594/1196] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:25:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:40.731587 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c364dd-98c2-49e0-980e-3c43da8ac036 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:40.815538 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:40.826716 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c364dd-98c2-49e0-980e-3c43da8ac036 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:40.859572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53533d3-fba4-4e3e-b662-32079885cd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:40.860181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53533d3-fba4-4e3e-b662-32079885cd16 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:40.860456 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53533d3-fba4-4e3e-b662-32079885cd16 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:40.860724 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53533d3-fba4-4e3e-b662-32079885cd16 None admin] PATH_INFO: `/v3/users/dc27c428933a4e34bdd3fedfba9f507f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:40.861828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53533d3-fba4-4e3e-b662-32079885cd16 None admin] RBAC: Authorizing `identity:get_user(user_id=dc27c428933a4e34bdd3fedfba9f507f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:40.863244 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53533d3-fba4-4e3e-b662-32079885cd16 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:40.864773 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 603/1197] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 08:25:40 2022] GET /identity/v3/users/dc27c428933a4e34bdd3fedfba9f507f => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:40.870497 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d421a-660f-4402-9a78-81366f0efe76 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:40.966167 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d421a-660f-4402-9a78-81366f0efe76 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:40.999165 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9357f28-5b27-4ba0-a95a-b98329650a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:40.999790 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9357f28-5b27-4ba0-a95a-b98329650a29 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:41.000068 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9357f28-5b27-4ba0-a95a-b98329650a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:41.000337 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9357f28-5b27-4ba0-a95a-b98329650a29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:41.046737 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:41.065818 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9357f28-5b27-4ba0-a95a-b98329650a29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:41.067243 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9357f28-5b27-4ba0-a95a-b98329650a29 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:41.161234 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 595/1198] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:25:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 08:25:41.167143 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53533d3-fba4-4e3e-b662-32079885cd16 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:41.266412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53533d3-fba4-4e3e-b662-32079885cd16 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:41.298203 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8138c49-9def-4c55-8930-5a3aaf058d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:41.298861 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8138c49-9def-4c55-8930-5a3aaf058d2b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:41.299140 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8138c49-9def-4c55-8930-5a3aaf058d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:41.299410 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8138c49-9def-4c55-8930-5a3aaf058d2b None admin] PATH_INFO: `/v3/projects/99e686a7984c450dbd26cf1f8399d86a` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:41.300547 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8138c49-9def-4c55-8930-5a3aaf058d2b None admin] RBAC: Authorizing `identity:get_project(project_id=99e686a7984c450dbd26cf1f8399d86a)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:41.302000 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8138c49-9def-4c55-8930-5a3aaf058d2b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:41.303662 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 604/1199] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 08:25:41 2022] GET /identity/v3/projects/99e686a7984c450dbd26cf1f8399d86a => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:41.309855 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9357f28-5b27-4ba0-a95a-b98329650a29 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:25:41.386658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:41.405587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9357f28-5b27-4ba0-a95a-b98329650a29 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:25:41.439094 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7fbc5d-0eeb-4f97-95a3-e12e3cd9ada4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:25:41.439688 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fbc5d-0eeb-4f97-95a3-e12e3cd9ada4 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:25:41.439963 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fbc5d-0eeb-4f97-95a3-e12e3cd9ada4 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:25:41.440226 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fbc5d-0eeb-4f97-95a3-e12e3cd9ada4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:25:41.440936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7fbc5d-0eeb-4f97-95a3-e12e3cd9ada4 None admin] RBAC: Adding query filter params (role.id=c1e514a4b5424f959b069f36cbfd8147, scope.project.id=99e686a7984c450dbd26cf1f8399d86a, user.id=dc27c428933a4e34bdd3fedfba9f507f) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:25:41.441274 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7fbc5d-0eeb-4f97-95a3-e12e3cd9ada4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:25:41.442665 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7fbc5d-0eeb-4f97-95a3-e12e3cd9ada4 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:25:41.448207 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 596/1200] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:25:41 2022] GET /identity/v3/role_assignments?user.id=dc27c428933a4e34bdd3fedfba9f507f&scope.project.id=99e686a7984c450dbd26cf1f8399d86a&role.id=c1e514a4b5424f959b069f36cbfd8147 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:25:51.294018 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=88 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:25:51.445702 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=28 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:26:50.354030 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc155f27-f75b-4892-a68a-36dbd0a931b0 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:50.354643 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc155f27-f75b-4892-a68a-36dbd0a931b0 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:50.354643 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc155f27-f75b-4892-a68a-36dbd0a931b0 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:50.355969 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 605/1201] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:26:50.361288 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706dbcb-a398-471e-8765-b94e88d44fab None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:50.361738 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706dbcb-a398-471e-8765-b94e88d44fab None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:50.361906 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706dbcb-a398-471e-8765-b94e88d44fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:50.380496 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-e706dbcb-a398-471e-8765-b94e88d44fab None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:26:50.381717 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e706dbcb-a398-471e-8765-b94e88d44fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:26:50.451603 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 597/1202] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:26:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:26:50.476088 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc155f27-f75b-4892-a68a-36dbd0a931b0 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:26:50.574972 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc155f27-f75b-4892-a68a-36dbd0a931b0 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:26:50.609083 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9011e66-5b72-479a-835f-af8b3b63833d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:26:50.609673 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9011e66-5b72-479a-835f-af8b3b63833d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:50.609952 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9011e66-5b72-479a-835f-af8b3b63833d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:50.610221 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9011e66-5b72-479a-835f-af8b3b63833d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:50.620621 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9011e66-5b72-479a-835f-af8b3b63833d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:26:50.623998 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9011e66-5b72-479a-835f-af8b3b63833d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:26:50.636133 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-c9011e66-5b72-479a-835f-af8b3b63833d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:26:50.636133 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:26:50.636133 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:26:50.636133 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:26:50.636133 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:26:50.636133 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:26:50.636133 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:26:50.636133 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:26:50.636133 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:26:50.636133 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:26:50.636133 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:26:50.636133 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:26:50.636133 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:26:50.636133 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:26:50.636133 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:26:50.636133 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:26:50.637361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:26:50.637361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:26:50.637361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:26:50.637361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:26:50.637361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:26:50.637361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:26:50.637361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:26:50.637361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:26:50.637361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:26:50.637361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:26:50.637361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:26:50.637361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:26:50.637361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:26:50.637361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:26:50.637361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:26:50.637361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:26:50.637361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:26:50.638353 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:26:50.638353 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:26:50.638353 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:26:50.638353 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 606/1203] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:26:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:26:50.643299 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e706dbcb-a398-471e-8765-b94e88d44fab None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:26:50.742245 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e706dbcb-a398-471e-8765-b94e88d44fab None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:26:50.770218 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:26:50.775842 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69371f1c-e0d6-4514-9225-96ee829347d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:26:50.776441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69371f1c-e0d6-4514-9225-96ee829347d3 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:50.776745 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69371f1c-e0d6-4514-9225-96ee829347d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:50.777020 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69371f1c-e0d6-4514-9225-96ee829347d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:50.777689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69371f1c-e0d6-4514-9225-96ee829347d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:26:50.777994 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69371f1c-e0d6-4514-9225-96ee829347d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:26:50.779376 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69371f1c-e0d6-4514-9225-96ee829347d3 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:26:50.790926 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 598/1204] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:26:50.796850 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9011e66-5b72-479a-835f-af8b3b63833d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:26:50.851235 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:26:50.898284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9011e66-5b72-479a-835f-af8b3b63833d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:26:50.930562 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2d5af-92bb-4d56-a4b9-ce7a8d2816da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:26:50.931151 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2d5af-92bb-4d56-a4b9-ce7a8d2816da None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:50.931427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2d5af-92bb-4d56-a4b9-ce7a8d2816da None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:50.931694 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2d5af-92bb-4d56-a4b9-ce7a8d2816da None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:50.932329 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2d5af-92bb-4d56-a4b9-ce7a8d2816da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:26:50.933926 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2d5af-92bb-4d56-a4b9-ce7a8d2816da None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:26:51.003515 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 607/1205] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 08:26:50 2022] POST /identity/v3/users => generated 251 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:26:53.044839 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38678667-24ed-47f1-8e6b-190ab06b648d None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:53.045419 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38678667-24ed-47f1-8e6b-190ab06b648d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:53.045419 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38678667-24ed-47f1-8e6b-190ab06b648d None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:53.046769 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 599/1206] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:26:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:26:53.052420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d7970-899c-4f16-abd1-209605a44c69 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:53.052884 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d7970-899c-4f16-abd1-209605a44c69 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:53.052991 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d7970-899c-4f16-abd1-209605a44c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:53.071605 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-861d7970-899c-4f16-abd1-209605a44c69 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:26:53.075077 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-861d7970-899c-4f16-abd1-209605a44c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:26:53.138202 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 608/1207] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:26:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:26:53.165971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38678667-24ed-47f1-8e6b-190ab06b648d None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:26:53.266440 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38678667-24ed-47f1-8e6b-190ab06b648d None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:26:53.299353 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bde03-0c69-4946-9125-2cbdadad8fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:26:53.299931 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bde03-0c69-4946-9125-2cbdadad8fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:53.300221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bde03-0c69-4946-9125-2cbdadad8fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:53.300487 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bde03-0c69-4946-9125-2cbdadad8fd8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:53.310693 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437bde03-0c69-4946-9125-2cbdadad8fd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:26:53.315261 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-437bde03-0c69-4946-9125-2cbdadad8fd8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:26:53.315261 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:26:53.315261 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:26:53.315261 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:26:53.315261 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:26:53.315261 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:26:53.315261 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:26:53.315261 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:26:53.315261 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:26:53.315261 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:26:53.315261 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:26:53.315261 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:26:53.315261 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:26:53.315261 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:26:53.315261 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:26:53.315261 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:26:53.316949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:26:53.316949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:26:53.316949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:26:53.316949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:26:53.316949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:26:53.316949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:26:53.316949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:26:53.316949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:26:53.316949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:26:53.316949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:26:53.316949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:26:53.316949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:26:53.316949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:26:53.316949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:26:53.316949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:26:53.316949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:26:53.317949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:26:53.317949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:26:53.317949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:26:53.317949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:26:53.317949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:26:53.317949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:26:53.317949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:26:53.317949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:26:53.317949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:26:53.317949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:26:53.317949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:26:53.317949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:26:53.317949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 600/1208] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 08:26:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:26:53.323041 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d7970-899c-4f16-abd1-209605a44c69 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:26:53.361882 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:26:53.421555 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d7970-899c-4f16-abd1-209605a44c69 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:26:53.454840 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb01c6-61c9-4c65-8b79-41635c1f6570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:26:53.455441 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb01c6-61c9-4c65-8b79-41635c1f6570 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:53.455715 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb01c6-61c9-4c65-8b79-41635c1f6570 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:53.455982 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb01c6-61c9-4c65-8b79-41635c1f6570 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:53.457270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb01c6-61c9-4c65-8b79-41635c1f6570 None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:26:53.457568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb01c6-61c9-4c65-8b79-41635c1f6570 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:26:53.458994 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb01c6-61c9-4c65-8b79-41635c1f6570 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:26:53.466819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 609/1209] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 08:26:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:26:53.472821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bde03-0c69-4946-9125-2cbdadad8fd8 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:26:53.521158 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:26:53.569736 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bde03-0c69-4946-9125-2cbdadad8fd8 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:26:53.604464 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae685082-1814-48b3-9ac9-cb314e210a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:26:53.606406 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae685082-1814-48b3-9ac9-cb314e210a86 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:53.606406 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae685082-1814-48b3-9ac9-cb314e210a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:53.606406 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae685082-1814-48b3-9ac9-cb314e210a86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:53.615842 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae685082-1814-48b3-9ac9-cb314e210a86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:26:53.617327 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae685082-1814-48b3-9ac9-cb314e210a86 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:26:53.630780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-ae685082-1814-48b3-9ac9-cb314e210a86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:26:53.630780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:26:53.630780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:26:53.630780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:26:53.630780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:26:53.630780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:26:53.630780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:26:53.630780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:26:53.630780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:26:53.630780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:26:53.630780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:26:53.630780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:26:53.630780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:26:53.630780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:26:53.630780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:26:53.630780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:26:53.636096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:26:53.636096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:26:53.636096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:26:53.636096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:26:53.636096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:26:53.636096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:26:53.636096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:26:53.636096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:26:53.636096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:26:53.636096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:26:53.636096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:26:53.636096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:26:53.636096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:26:53.636096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:26:53.636096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:26:53.636096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:26:53.636096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:26:53.637218 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:26:53.637218 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:26:53.637218 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:26:53.637218 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 601/1210] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:26:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:26:53.637611 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb01c6-61c9-4c65-8b79-41635c1f6570 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:26:53.734140 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:26:53.736976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb01c6-61c9-4c65-8b79-41635c1f6570 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:26:53.769846 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77430abc-666b-40d3-bdbe-1962d0b54139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:26:53.770441 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77430abc-666b-40d3-bdbe-1962d0b54139 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:53.770718 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77430abc-666b-40d3-bdbe-1962d0b54139 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:53.770987 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77430abc-666b-40d3-bdbe-1962d0b54139 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:53.771618 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77430abc-666b-40d3-bdbe-1962d0b54139 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:26:53.771915 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77430abc-666b-40d3-bdbe-1962d0b54139 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:26:53.773355 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77430abc-666b-40d3-bdbe-1962d0b54139 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:26:53.785361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 610/1211] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:26:53.791349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae685082-1814-48b3-9ac9-cb314e210a86 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:26:53.889433 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae685082-1814-48b3-9ac9-cb314e210a86 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:26:53.890947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:26:53.923286 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8afbb82-9267-46a4-9c2a-ceae8eb930e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:26:53.923911 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8afbb82-9267-46a4-9c2a-ceae8eb930e9 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:53.924242 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8afbb82-9267-46a4-9c2a-ceae8eb930e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:53.924512 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8afbb82-9267-46a4-9c2a-ceae8eb930e9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:53.931821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8afbb82-9267-46a4-9c2a-ceae8eb930e9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:26:53.933369 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8afbb82-9267-46a4-9c2a-ceae8eb930e9 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:26:53.944494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-b8afbb82-9267-46a4-9c2a-ceae8eb930e9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 08:26:53.944494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:26:53.944494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:26:53.944494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:26:53.944494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:26:53.944494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:26:53.944494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:26:53.944494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:26:53.944494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:26:53.944494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:26:53.944494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:26:53.944494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:26:53.944494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:26:53.944494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:26:53.944494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:26:53.944494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:26:53.945473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:26:53.945473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:26:53.945473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:26:53.945473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:26:53.945473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:26:53.945473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:26:53.945473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:26:53.945473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:26:53.945473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:26:53.945473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:26:53.945473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:26:53.945473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:26:53.945473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:26:53.945473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:26:53.945473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:26:53.945473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:26:53.945473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:26:53.945473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:26:53.946483 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:26:53.946483 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:26:53.946483 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:26:53.946483 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:26:53.946483 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:26:53.946483 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:26:53.946483 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:26:53.946483 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:26:53.946483 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 08:26:53.946483 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:26:53.946483 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 602/1212] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:26:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:26:53.951405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77430abc-666b-40d3-bdbe-1962d0b54139 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:26:54.049044 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77430abc-666b-40d3-bdbe-1962d0b54139 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:26:54.082862 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd0b39-3350-4112-a38c-fbc711b6f600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:26:54.083493 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd0b39-3350-4112-a38c-fbc711b6f600 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:54.083724 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd0b39-3350-4112-a38c-fbc711b6f600 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:54.083993 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd0b39-3350-4112-a38c-fbc711b6f600 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:54.084717 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd0b39-3350-4112-a38c-fbc711b6f600 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:26:54.085021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd0b39-3350-4112-a38c-fbc711b6f600 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:26:54.086478 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd0b39-3350-4112-a38c-fbc711b6f600 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:26:54.103467 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 611/1213] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Aug 8 08:26:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:26:54.109506 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8afbb82-9267-46a4-9c2a-ceae8eb930e9 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:26:54.206059 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8afbb82-9267-46a4-9c2a-ceae8eb930e9 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:26:54.240771 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75813a52-d30c-492f-9c82-725070e479d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:26:54.241426 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75813a52-d30c-492f-9c82-725070e479d5 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:54.248576 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75813a52-d30c-492f-9c82-725070e479d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:54.248576 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75813a52-d30c-492f-9c82-725070e479d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:54.272632 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:26:54.309174 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75813a52-d30c-492f-9c82-725070e479d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:26:54.310617 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75813a52-d30c-492f-9c82-725070e479d5 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:26:54.405889 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 603/1214] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:26:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 08:26:54.412490 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd0b39-3350-4112-a38c-fbc711b6f600 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:26:54.431672 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:26:54.509375 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd0b39-3350-4112-a38c-fbc711b6f600 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:26:54.543519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ddbf95-0733-47a6-89e5-ef4a36629f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:26:54.544139 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ddbf95-0733-47a6-89e5-ef4a36629f9d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:54.544413 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ddbf95-0733-47a6-89e5-ef4a36629f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:54.544681 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ddbf95-0733-47a6-89e5-ef4a36629f9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:54.554429 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ddbf95-0733-47a6-89e5-ef4a36629f9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:26:54.555851 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ddbf95-0733-47a6-89e5-ef4a36629f9d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:26:54.569784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-95ddbf95-0733-47a6-89e5-ef4a36629f9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:26:54.569784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:26:54.569784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:26:54.569784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:26:54.569784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:26:54.569784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:26:54.569784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:26:54.569784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:26:54.569784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:26:54.569784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:26:54.569784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:26:54.569784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:26:54.569784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:26:54.569784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:26:54.569784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:26:54.569784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:26:54.570815 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:26:54.570815 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:26:54.570815 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:26:54.570815 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:26:54.570815 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:26:54.570815 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:26:54.570815 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:26:54.570815 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:26:54.570815 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:26:54.570815 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:26:54.570815 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:26:54.570815 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:26:54.570815 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:26:54.570815 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:26:54.570815 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:26:54.570815 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:26:54.570815 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:26:54.571706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:26:54.571706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:26:54.571706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:26:54.571706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 612/1215] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:26:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:26:54.576525 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75813a52-d30c-492f-9c82-725070e479d5 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:26:54.636628 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:26:54.674376 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75813a52-d30c-492f-9c82-725070e479d5 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:26:54.708855 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a71c64-7400-4ac3-8580-2bb00ad5ce30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:26:54.709475 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a71c64-7400-4ac3-8580-2bb00ad5ce30 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:54.709751 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a71c64-7400-4ac3-8580-2bb00ad5ce30 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:54.710019 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a71c64-7400-4ac3-8580-2bb00ad5ce30 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:54.710651 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a71c64-7400-4ac3-8580-2bb00ad5ce30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:26:54.711007 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a71c64-7400-4ac3-8580-2bb00ad5ce30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:26:54.712354 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a71c64-7400-4ac3-8580-2bb00ad5ce30 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:26:54.723557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 604/1216] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:26:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:26:54.729262 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ddbf95-0733-47a6-89e5-ef4a36629f9d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:26:54.796638 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:26:54.826480 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ddbf95-0733-47a6-89e5-ef4a36629f9d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:26:54.859660 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8df3c-5bdd-455c-af44-e7b9d230058f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:26:54.860273 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8df3c-5bdd-455c-af44-e7b9d230058f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:54.860546 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8df3c-5bdd-455c-af44-e7b9d230058f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:54.860812 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8df3c-5bdd-455c-af44-e7b9d230058f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:54.867368 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc8df3c-5bdd-455c-af44-e7b9d230058f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:26:54.868790 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc8df3c-5bdd-455c-af44-e7b9d230058f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:26:54.880110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-4fc8df3c-5bdd-455c-af44-e7b9d230058f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:26:54.880110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:26:54.880110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:26:54.880110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:26:54.880110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:26:54.880110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:26:54.880110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:26:54.880110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:26:54.880110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:26:54.880110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:26:54.880110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:26:54.880110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:26:54.880110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:26:54.880110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:26:54.880110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:26:54.881172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:26:54.881172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:26:54.881172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:26:54.881172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:26:54.881172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:26:54.881172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:26:54.881172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:26:54.881172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:26:54.881172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:26:54.881172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:26:54.881172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:26:54.881172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:26:54.881172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:26:54.881172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:26:54.881172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:26:54.881172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:26:54.881172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:26:54.882299 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:26:54.882299 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:26:54.882299 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:26:54.882299 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:26:54.882299 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:26:54.882299 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:26:54.882299 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:26:54.882299 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:26:54.882299 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 613/1217] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:26:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:26:54.887115 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a71c64-7400-4ac3-8580-2bb00ad5ce30 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:26:54.986141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a71c64-7400-4ac3-8580-2bb00ad5ce30 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:26:55.007821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:26:55.018718 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f94632-46cf-4867-8356-4c706fa16b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:26:55.019844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f94632-46cf-4867-8356-4c706fa16b85 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:55.020117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f94632-46cf-4867-8356-4c706fa16b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:55.020382 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f94632-46cf-4867-8356-4c706fa16b85 None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:55.021061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f94632-46cf-4867-8356-4c706fa16b85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:26:55.021380 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f94632-46cf-4867-8356-4c706fa16b85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:26:55.022787 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f94632-46cf-4867-8356-4c706fa16b85 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:26:55.034063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 605/1218] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:26:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:26:55.040325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8df3c-5bdd-455c-af44-e7b9d230058f None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:26:55.140052 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8df3c-5bdd-455c-af44-e7b9d230058f None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:26:55.166985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:26:55.172683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e48036-dca7-4583-971a-764e11696945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:26:55.173317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e48036-dca7-4583-971a-764e11696945 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:55.173596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e48036-dca7-4583-971a-764e11696945 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:55.173865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e48036-dca7-4583-971a-764e11696945 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:55.174579 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e48036-dca7-4583-971a-764e11696945 None admin] RBAC: Adding query filter params (role.id=402c3e6fe3364285bd72784a297db11e, scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4, user.id=98c23a782b9b4144bace1248c3b2e754) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:26:55.175430 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e48036-dca7-4583-971a-764e11696945 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:26:55.176881 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e48036-dca7-4583-971a-764e11696945 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:26:55.182266 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 614/1219] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:26:55 2022] GET /identity/v3/role_assignments?user.id=98c23a782b9b4144bace1248c3b2e754&scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4&role.id=402c3e6fe3364285bd72784a297db11e => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:26:57.172914 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6e80d-0a0a-45f0-96fb-3af4a579ed44 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:57.173495 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6e80d-0a0a-45f0-96fb-3af4a579ed44 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:57.173495 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6e80d-0a0a-45f0-96fb-3af4a579ed44 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:57.174856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 606/1220] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:26:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:26:57.180504 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727abbe2-0b58-441d-8bbf-90e5123adef4 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:57.180960 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727abbe2-0b58-441d-8bbf-90e5123adef4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:57.181053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727abbe2-0b58-441d-8bbf-90e5123adef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:57.199576 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-727abbe2-0b58-441d-8bbf-90e5123adef4 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:26:57.200753 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-727abbe2-0b58-441d-8bbf-90e5123adef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:26:57.270820 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 615/1221] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:26:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:26:57.293442 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa6e80d-0a0a-45f0-96fb-3af4a579ed44 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:26:57.394756 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa6e80d-0a0a-45f0-96fb-3af4a579ed44 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:26:57.429735 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac39e1a-9e68-41d8-9f47-6bd8b1b8d441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:26:57.430303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac39e1a-9e68-41d8-9f47-6bd8b1b8d441 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:57.430579 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac39e1a-9e68-41d8-9f47-6bd8b1b8d441 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:57.430849 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac39e1a-9e68-41d8-9f47-6bd8b1b8d441 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:57.441609 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac39e1a-9e68-41d8-9f47-6bd8b1b8d441 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:26:57.450984 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-9ac39e1a-9e68-41d8-9f47-6bd8b1b8d441 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:26:57.450984 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:26:57.450984 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:26:57.450984 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:26:57.450984 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:26:57.450984 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:26:57.450984 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:26:57.450984 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:26:57.450984 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:26:57.450984 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:26:57.450984 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:26:57.450984 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:26:57.450984 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:26:57.450984 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:26:57.450984 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:26:57.450984 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:26:57.452150 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:26:57.452150 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:26:57.452150 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:26:57.452150 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:26:57.452150 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:26:57.452150 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:26:57.452150 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:26:57.452150 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:26:57.452150 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:26:57.452150 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:26:57.452150 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:26:57.452150 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:26:57.452150 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:26:57.452150 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:26:57.452150 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:26:57.452150 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:26:57.453008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:26:57.453008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:26:57.453008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:26:57.453008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:26:57.453008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:26:57.453008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:26:57.453008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:26:57.453008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:26:57.453008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:26:57.453008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:26:57.453008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:26:57.453008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:26:57.453008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 607/1222] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 08:26:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:26:57.453939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727abbe2-0b58-441d-8bbf-90e5123adef4 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:26:57.549729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727abbe2-0b58-441d-8bbf-90e5123adef4 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:26:57.574249 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:26:57.584707 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213b5ee-3f7c-4a4c-b3c4-636bdd2ec472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:26:57.585546 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213b5ee-3f7c-4a4c-b3c4-636bdd2ec472 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:57.585823 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213b5ee-3f7c-4a4c-b3c4-636bdd2ec472 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:57.586092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213b5ee-3f7c-4a4c-b3c4-636bdd2ec472 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:57.586746 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213b5ee-3f7c-4a4c-b3c4-636bdd2ec472 None admin] RBAC: Adding query filter params (name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:26:57.587042 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213b5ee-3f7c-4a4c-b3c4-636bdd2ec472 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:26:57.588398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213b5ee-3f7c-4a4c-b3c4-636bdd2ec472 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:26:57.596472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 616/1223] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 08:26:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:26:57.602623 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac39e1a-9e68-41d8-9f47-6bd8b1b8d441 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:26:57.647287 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:26:57.702676 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac39e1a-9e68-41d8-9f47-6bd8b1b8d441 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:26:57.736273 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ed95f-312f-4e0c-afd1-38aacf468b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:26:57.736890 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ed95f-312f-4e0c-afd1-38aacf468b7f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:57.737189 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ed95f-312f-4e0c-afd1-38aacf468b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:57.738009 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ed95f-312f-4e0c-afd1-38aacf468b7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:57.747732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839ed95f-312f-4e0c-afd1-38aacf468b7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:26:57.749178 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839ed95f-312f-4e0c-afd1-38aacf468b7f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:26:57.763478 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-839ed95f-312f-4e0c-afd1-38aacf468b7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:26:57.763478 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:26:57.763478 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:26:57.763478 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:26:57.763478 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:26:57.763478 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:26:57.763478 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:26:57.763478 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:26:57.763478 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:26:57.763478 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:26:57.763478 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:26:57.763478 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:26:57.763478 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:26:57.763478 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:26:57.763478 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:26:57.763478 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:26:57.764848 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:26:57.764848 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:26:57.764848 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:26:57.764848 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:26:57.764848 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:26:57.764848 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:26:57.764848 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:26:57.764848 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:26:57.764848 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:26:57.764848 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:26:57.764848 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:26:57.764848 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:26:57.764848 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:26:57.764848 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:26:57.764848 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:26:57.764848 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:26:57.764848 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:26:57.765789 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:26:57.765789 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:26:57.765789 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:26:57.765789 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 608/1224] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:26:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:26:57.770544 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213b5ee-3f7c-4a4c-b3c4-636bdd2ec472 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:26:57.871268 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213b5ee-3f7c-4a4c-b3c4-636bdd2ec472 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:26:57.903777 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c317b2-81bb-478a-bdcd-6d8b5716f3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:26:57.904373 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c317b2-81bb-478a-bdcd-6d8b5716f3d3 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:57.904649 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c317b2-81bb-478a-bdcd-6d8b5716f3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:57.904919 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c317b2-81bb-478a-bdcd-6d8b5716f3d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:57.905652 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c317b2-81bb-478a-bdcd-6d8b5716f3d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:26:57.905960 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c317b2-81bb-478a-bdcd-6d8b5716f3d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:26:57.907316 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c317b2-81bb-478a-bdcd-6d8b5716f3d3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:26:57.920335 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 617/1225] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:26:57.926185 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ed95f-312f-4e0c-afd1-38aacf468b7f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:26:58.020065 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:26:58.026505 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ed95f-312f-4e0c-afd1-38aacf468b7f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:26:58.059291 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f406aa65-add7-4aaf-8c12-9201a5bcaa3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:26:58.059910 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406aa65-add7-4aaf-8c12-9201a5bcaa3c None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:58.060187 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406aa65-add7-4aaf-8c12-9201a5bcaa3c None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:58.060457 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406aa65-add7-4aaf-8c12-9201a5bcaa3c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:58.068217 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f406aa65-add7-4aaf-8c12-9201a5bcaa3c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:26:58.069680 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f406aa65-add7-4aaf-8c12-9201a5bcaa3c None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:26:58.080297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-f406aa65-add7-4aaf-8c12-9201a5bcaa3c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 08:26:58.080297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:26:58.080297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:26:58.080297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:26:58.080297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:26:58.080297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:26:58.080297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:26:58.080297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:26:58.080297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:26:58.080297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:26:58.080297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:26:58.080297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:26:58.080297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:26:58.080297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:26:58.080297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:26:58.080297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:26:58.081387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:26:58.081387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:26:58.081387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:26:58.081387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:26:58.081387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:26:58.081387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:26:58.081387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:26:58.081387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:26:58.081387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:26:58.081387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:26:58.081387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:26:58.081387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:26:58.081387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:26:58.081387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:26:58.081387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:26:58.081387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:26:58.081387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:26:58.081387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:26:58.082402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:26:58.082402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:26:58.082402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:26:58.082402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:26:58.082402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:26:58.082402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:26:58.082402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:26:58.082402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:26:58.082402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 08:26:58.082402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:26:58.082402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 609/1226] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:26:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:26:58.087310 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c317b2-81bb-478a-bdcd-6d8b5716f3d3 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:26:58.134199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:26:58.184072 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c317b2-81bb-478a-bdcd-6d8b5716f3d3 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:26:58.219107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c95d2b-9857-49e9-bea2-5a6d7df61bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:26:58.219767 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c95d2b-9857-49e9-bea2-5a6d7df61bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:58.220047 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c95d2b-9857-49e9-bea2-5a6d7df61bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:58.220319 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c95d2b-9857-49e9-bea2-5a6d7df61bf5 None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:58.221042 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c95d2b-9857-49e9-bea2-5a6d7df61bf5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:26:58.221379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c95d2b-9857-49e9-bea2-5a6d7df61bf5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:26:58.222930 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c95d2b-9857-49e9-bea2-5a6d7df61bf5 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:26:58.239888 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 618/1227] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Aug 8 08:26:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:26:58.245584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f406aa65-add7-4aaf-8c12-9201a5bcaa3c None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:26:58.345498 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f406aa65-add7-4aaf-8c12-9201a5bcaa3c None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:26:58.381393 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde685c-8f1f-4959-8ee9-74654e07a1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:26:58.382007 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde685c-8f1f-4959-8ee9-74654e07a1f1 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:58.382278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde685c-8f1f-4959-8ee9-74654e07a1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:58.382542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde685c-8f1f-4959-8ee9-74654e07a1f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:58.392624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde685c-8f1f-4959-8ee9-74654e07a1f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:26:58.396886 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde685c-8f1f-4959-8ee9-74654e07a1f1 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:26:58.409776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-dbde685c-8f1f-4959-8ee9-74654e07a1f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:26:58.409776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:26:58.409776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:26:58.409776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:26:58.409776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:26:58.409776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:26:58.409776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:26:58.409776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:26:58.409776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:26:58.409776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:26:58.409776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:26:58.409776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:26:58.409776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:26:58.409776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:26:58.409776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:26:58.409776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:26:58.410904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:26:58.410904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:26:58.410904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:26:58.410904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:26:58.410904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:26:58.410904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:26:58.410904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:26:58.410904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:26:58.410904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:26:58.410904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:26:58.410904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:26:58.410904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:26:58.410904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:26:58.410904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:26:58.410904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:26:58.410904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:26:58.410904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:26:58.413554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:26:58.413554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:26:58.413554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:26:58.413554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 610/1228] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:26:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:26:58.416848 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c95d2b-9857-49e9-bea2-5a6d7df61bf5 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:26:58.514595 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c95d2b-9857-49e9-bea2-5a6d7df61bf5 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:26:58.516880 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:26:58.548058 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdce7f0-4922-4799-9778-4b93f8185a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:26:58.548742 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdce7f0-4922-4799-9778-4b93f8185a8e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:58.549015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdce7f0-4922-4799-9778-4b93f8185a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:58.549312 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdce7f0-4922-4799-9778-4b93f8185a8e None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:58.549964 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdce7f0-4922-4799-9778-4b93f8185a8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:26:58.550263 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdce7f0-4922-4799-9778-4b93f8185a8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:26:58.551617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdce7f0-4922-4799-9778-4b93f8185a8e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:26:58.562405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 619/1229] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:26:58.568657 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde685c-8f1f-4959-8ee9-74654e07a1f1 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:26:58.577830 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:26:58.666303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde685c-8f1f-4959-8ee9-74654e07a1f1 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:26:58.700129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d7056-90db-40c2-a67b-a24d2cce5e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:26:58.700775 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d7056-90db-40c2-a67b-a24d2cce5e07 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:58.701052 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d7056-90db-40c2-a67b-a24d2cce5e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:58.701366 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d7056-90db-40c2-a67b-a24d2cce5e07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:58.708711 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d7056-90db-40c2-a67b-a24d2cce5e07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:26:58.710167 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d7056-90db-40c2-a67b-a24d2cce5e07 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:26:58.721336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-553d7056-90db-40c2-a67b-a24d2cce5e07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:26:58.721336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:26:58.721336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:26:58.721336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:26:58.721336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:26:58.721336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:26:58.721336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:26:58.721336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:26:58.721336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:26:58.721336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:26:58.721336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:26:58.721336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:26:58.721336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:26:58.721336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:26:58.721336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:26:58.727430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:26:58.727430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:26:58.727430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:26:58.727430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:26:58.727430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:26:58.727430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:26:58.727430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:26:58.727430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:26:58.727430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:26:58.727430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:26:58.727430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:26:58.727430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:26:58.727430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:26:58.727430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:26:58.727430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:26:58.727430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:26:58.727430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:26:58.730518 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:26:58.730518 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:26:58.730518 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:26:58.730518 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:26:58.730518 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:26:58.730518 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:26:58.730518 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:26:58.730518 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:26:58.730518 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 611/1230] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:26:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:26:58.731126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdce7f0-4922-4799-9778-4b93f8185a8e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:26:58.826212 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdce7f0-4922-4799-9778-4b93f8185a8e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:26:58.858472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b5a18-febf-477f-be6a-003c65b5f731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:26:58.859056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b5a18-febf-477f-be6a-003c65b5f731 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:58.859336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b5a18-febf-477f-be6a-003c65b5f731 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:58.859603 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b5a18-febf-477f-be6a-003c65b5f731 None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:58.860281 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b5a18-febf-477f-be6a-003c65b5f731 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:26:58.860589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b5a18-febf-477f-be6a-003c65b5f731 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:26:58.862031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b5a18-febf-477f-be6a-003c65b5f731 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:26:58.874144 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 620/1231] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:26:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:26:58.880724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d7056-90db-40c2-a67b-a24d2cce5e07 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:26:58.944888 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:26:58.978407 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d7056-90db-40c2-a67b-a24d2cce5e07 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:26:59.011463 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb30604-b656-410c-b8f3-5765c832eba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:26:59.012107 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb30604-b656-410c-b8f3-5765c832eba6 None admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:26:59.012386 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb30604-b656-410c-b8f3-5765c832eba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:26:59.012663 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb30604-b656-410c-b8f3-5765c832eba6 None admin] PATH_INFO: `/v3/projects/71e5e82c9f454cdd91dfeac5a57968c4/users/98c23a782b9b4144bace1248c3b2e754/roles/402c3e6fe3364285bd72784a297db11e` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:26:59.027190 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb30604-b656-410c-b8f3-5765c832eba6 None admin] RBAC: Authorizing `identity:create_grant(project_id=71e5e82c9f454cdd91dfeac5a57968c4, user_id=98c23a782b9b4144bace1248c3b2e754, role_id=402c3e6fe3364285bd72784a297db11e)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:26:59.028601 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb30604-b656-410c-b8f3-5765c832eba6 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:26:59.083250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 612/1232] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 08:26:58 2022] PUT /identity/v3/projects/71e5e82c9f454cdd91dfeac5a57968c4/users/98c23a782b9b4144bace1248c3b2e754/roles/402c3e6fe3364285bd72784a297db11e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:27:01.046731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad8f3f-42c0-4e2c-aa08-e9d6602e1ec6 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:01.047280 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad8f3f-42c0-4e2c-aa08-e9d6602e1ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:01.047280 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad8f3f-42c0-4e2c-aa08-e9d6602e1ec6 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:01.048629 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 621/1233] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:27:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:27:01.054311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b589e71-51d9-4365-9f13-4962485ae3ec None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:01.054752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b589e71-51d9-4365-9f13-4962485ae3ec None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:01.054861 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b589e71-51d9-4365-9f13-4962485ae3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:01.073004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-1b589e71-51d9-4365-9f13-4962485ae3ec None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:27:01.074202 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1b589e71-51d9-4365-9f13-4962485ae3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:27:01.138733 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 613/1234] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:27:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:27:01.161519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad8f3f-42c0-4e2c-aa08-e9d6602e1ec6 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:01.186341 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:01.262657 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad8f3f-42c0-4e2c-aa08-e9d6602e1ec6 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:01.295402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a8846e-09be-4c58-93ab-7898217eb977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:01.296069 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a8846e-09be-4c58-93ab-7898217eb977 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:01.296346 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a8846e-09be-4c58-93ab-7898217eb977 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:01.296615 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a8846e-09be-4c58-93ab-7898217eb977 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:01.307455 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a8846e-09be-4c58-93ab-7898217eb977 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:27:01.312197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-e7a8846e-09be-4c58-93ab-7898217eb977 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:27:01.312197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:27:01.312197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:27:01.312197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:27:01.312197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:27:01.312197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:27:01.312197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:27:01.312197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:27:01.312197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:27:01.312197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:27:01.312197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:27:01.312197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:27:01.312197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:27:01.312197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:27:01.312197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:27:01.312197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:27:01.313261 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:27:01.313261 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:27:01.313261 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:27:01.313261 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:27:01.313261 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:27:01.313261 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:27:01.313261 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:27:01.313261 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:27:01.313261 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:27:01.313261 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:27:01.313261 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:27:01.313261 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:27:01.313261 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:27:01.313261 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:27:01.313261 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:27:01.313261 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:27:01.314108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:27:01.314108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:27:01.314108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:27:01.314108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:27:01.314108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:27:01.314108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:27:01.314108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:27:01.314108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:27:01.314108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:27:01.314108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:27:01.314108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 08:27:01.314108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:27:01.314108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 622/1235] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 08:27:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:27:01.319890 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b589e71-51d9-4365-9f13-4962485ae3ec None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:01.365686 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:01.418980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b589e71-51d9-4365-9f13-4962485ae3ec None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:01.452944 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444c64c-0dfc-4e8c-b16d-e5cb2687a1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:01.453597 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444c64c-0dfc-4e8c-b16d-e5cb2687a1bd None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:01.453958 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444c64c-0dfc-4e8c-b16d-e5cb2687a1bd None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:01.454232 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444c64c-0dfc-4e8c-b16d-e5cb2687a1bd None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:01.454937 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444c64c-0dfc-4e8c-b16d-e5cb2687a1bd None admin] RBAC: Adding query filter params (name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:27:01.455235 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444c64c-0dfc-4e8c-b16d-e5cb2687a1bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:01.456752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444c64c-0dfc-4e8c-b16d-e5cb2687a1bd None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:01.464522 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 614/1236] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 08:27:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:27:01.470838 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a8846e-09be-4c58-93ab-7898217eb977 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:01.556239 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:01.571323 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a8846e-09be-4c58-93ab-7898217eb977 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:01.604016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cbd77d-91aa-4b77-bbe7-fe99778e3906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:01.604636 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbd77d-91aa-4b77-bbe7-fe99778e3906 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:01.604914 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbd77d-91aa-4b77-bbe7-fe99778e3906 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:01.605210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbd77d-91aa-4b77-bbe7-fe99778e3906 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:01.615305 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cbd77d-91aa-4b77-bbe7-fe99778e3906 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:01.616707 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cbd77d-91aa-4b77-bbe7-fe99778e3906 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:01.631095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-e8cbd77d-91aa-4b77-bbe7-fe99778e3906 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:27:01.631095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:27:01.631095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:27:01.631095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:27:01.631095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:27:01.631095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:27:01.631095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:27:01.631095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:27:01.631095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:27:01.631095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:27:01.631095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:27:01.631095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:27:01.631095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:27:01.631095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:27:01.631095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:27:01.631095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:27:01.632179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:27:01.632179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:27:01.632179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:27:01.632179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:27:01.632179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:27:01.632179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:27:01.632179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:27:01.632179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:27:01.632179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:27:01.632179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:27:01.632179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:27:01.632179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:27:01.632179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:27:01.632179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:27:01.632179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:27:01.632179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:27:01.632179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:27:01.633448 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:27:01.633448 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:27:01.633448 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:27:01.633448 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 623/1237] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:27:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:27:01.637966 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444c64c-0dfc-4e8c-b16d-e5cb2687a1bd None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:01.736241 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444c64c-0dfc-4e8c-b16d-e5cb2687a1bd None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:01.745582 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:01.770786 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f9b02-6bac-422a-99b7-40ae45885b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:01.771416 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f9b02-6bac-422a-99b7-40ae45885b6f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:01.771698 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f9b02-6bac-422a-99b7-40ae45885b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:01.771970 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f9b02-6bac-422a-99b7-40ae45885b6f None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:01.772673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f9b02-6bac-422a-99b7-40ae45885b6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:27:01.773094 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f9b02-6bac-422a-99b7-40ae45885b6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:01.774630 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f9b02-6bac-422a-99b7-40ae45885b6f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:01.785921 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 615/1238] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:27:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:27:01.791703 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cbd77d-91aa-4b77-bbe7-fe99778e3906 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:01.894239 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cbd77d-91aa-4b77-bbe7-fe99778e3906 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:01.926711 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398877b-acdc-4681-8f33-7e420b96614b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:01.927328 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398877b-acdc-4681-8f33-7e420b96614b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:01.927602 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398877b-acdc-4681-8f33-7e420b96614b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:01.927875 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398877b-acdc-4681-8f33-7e420b96614b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:01.930519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:01.936083 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398877b-acdc-4681-8f33-7e420b96614b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:01.937618 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398877b-acdc-4681-8f33-7e420b96614b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:01.949279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-5398877b-acdc-4681-8f33-7e420b96614b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 08:27:01.949279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:27:01.949279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:27:01.949279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:27:01.949279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:27:01.949279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:27:01.949279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:27:01.949279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:27:01.949279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:27:01.949279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:27:01.949279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:27:01.949279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:27:01.949279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:27:01.949279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:27:01.949279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:27:01.949279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:27:01.950482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:27:01.950482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:27:01.950482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:27:01.950482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:27:01.950482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:27:01.950482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:27:01.950482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:27:01.950482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:27:01.950482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:27:01.950482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:27:01.950482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:27:01.950482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:27:01.950482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:27:01.950482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:27:01.950482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:27:01.950482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:27:01.950482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:27:01.950482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:27:01.951718 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:27:01.951718 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:27:01.951718 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:27:01.951718 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:27:01.951718 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:27:01.951718 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:27:01.951718 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:27:01.951718 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:27:01.951718 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 08:27:01.951718 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:27:01.951718 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 624/1239] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:27:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:27:01.956317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f9b02-6bac-422a-99b7-40ae45885b6f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:02.054931 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f9b02-6bac-422a-99b7-40ae45885b6f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:02.087779 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71586ed-69b6-44e9-8751-f6e61268273f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:02.088370 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71586ed-69b6-44e9-8751-f6e61268273f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:02.088649 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71586ed-69b6-44e9-8751-f6e61268273f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:02.088917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71586ed-69b6-44e9-8751-f6e61268273f None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:02.089665 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71586ed-69b6-44e9-8751-f6e61268273f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:27:02.089967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71586ed-69b6-44e9-8751-f6e61268273f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:02.091660 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71586ed-69b6-44e9-8751-f6e61268273f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:02.108350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 616/1240] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Aug 8 08:27:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:27:02.114653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398877b-acdc-4681-8f33-7e420b96614b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:02.213584 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398877b-acdc-4681-8f33-7e420b96614b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:02.247015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea50a92-3e2a-40b3-81fc-74b03acde00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:02.247612 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea50a92-3e2a-40b3-81fc-74b03acde00a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:02.247900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea50a92-3e2a-40b3-81fc-74b03acde00a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:02.248208 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea50a92-3e2a-40b3-81fc-74b03acde00a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:02.308084 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:02.314422 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea50a92-3e2a-40b3-81fc-74b03acde00a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:02.316387 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea50a92-3e2a-40b3-81fc-74b03acde00a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:02.417293 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 625/1241] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:27:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 08:27:02.423037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71586ed-69b6-44e9-8751-f6e61268273f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:02.450427 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:02.522662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71586ed-69b6-44e9-8751-f6e61268273f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:02.555736 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd6ef5-280d-45f3-bea5-e7c83a259552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:02.556404 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd6ef5-280d-45f3-bea5-e7c83a259552 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:02.556680 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd6ef5-280d-45f3-bea5-e7c83a259552 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:02.556947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd6ef5-280d-45f3-bea5-e7c83a259552 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:02.567270 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd6ef5-280d-45f3-bea5-e7c83a259552 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:02.568711 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd6ef5-280d-45f3-bea5-e7c83a259552 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:02.582679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-d8cd6ef5-280d-45f3-bea5-e7c83a259552 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:27:02.582679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:27:02.582679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:27:02.582679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:27:02.582679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:27:02.582679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:27:02.582679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:27:02.582679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:27:02.582679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:27:02.582679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:27:02.582679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:27:02.582679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:27:02.582679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:27:02.582679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:27:02.582679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:27:02.582679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:27:02.583812 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:27:02.583812 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:27:02.583812 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:27:02.583812 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:27:02.583812 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:27:02.583812 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:27:02.583812 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:27:02.583812 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:27:02.583812 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:27:02.583812 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:27:02.583812 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:27:02.583812 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:27:02.583812 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:27:02.583812 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:27:02.583812 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:27:02.583812 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:27:02.583812 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:27:02.584759 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:27:02.584759 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:27:02.584759 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:27:02.584759 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 617/1242] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:27:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:27:02.589654 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea50a92-3e2a-40b3-81fc-74b03acde00a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:02.680561 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:02.686211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea50a92-3e2a-40b3-81fc-74b03acde00a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:02.718674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe1807-bcd7-43a3-9bde-8bc46fdc61cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:02.719264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe1807-bcd7-43a3-9bde-8bc46fdc61cb None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:02.719539 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe1807-bcd7-43a3-9bde-8bc46fdc61cb None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:02.719807 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe1807-bcd7-43a3-9bde-8bc46fdc61cb None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:02.720437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbe1807-bcd7-43a3-9bde-8bc46fdc61cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:27:02.720732 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbe1807-bcd7-43a3-9bde-8bc46fdc61cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:02.722151 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbe1807-bcd7-43a3-9bde-8bc46fdc61cb None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:02.732790 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 626/1243] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:27:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:27:02.739115 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd6ef5-280d-45f3-bea5-e7c83a259552 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:02.819381 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:02.840065 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd6ef5-280d-45f3-bea5-e7c83a259552 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:02.873014 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e81163-80f7-4c78-9b9b-d50ec0ba32eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:02.877459 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e81163-80f7-4c78-9b9b-d50ec0ba32eb None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:02.877459 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e81163-80f7-4c78-9b9b-d50ec0ba32eb None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:02.877459 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e81163-80f7-4c78-9b9b-d50ec0ba32eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:02.881122 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e81163-80f7-4c78-9b9b-d50ec0ba32eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:02.882582 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e81163-80f7-4c78-9b9b-d50ec0ba32eb None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:02.894129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-73e81163-80f7-4c78-9b9b-d50ec0ba32eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:27:02.894129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:27:02.894129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:27:02.894129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:27:02.894129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:27:02.894129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:27:02.894129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:27:02.894129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:27:02.894129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:27:02.894129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:27:02.894129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:27:02.894129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:27:02.894129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:27:02.894129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:27:02.894129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:27:02.895099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:27:02.895099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:27:02.895099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:27:02.895099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:27:02.895099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:27:02.895099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:27:02.895099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:27:02.895099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:27:02.895099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:27:02.895099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:27:02.895099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:27:02.895099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:27:02.895099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:27:02.895099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:27:02.895099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:27:02.895099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:27:02.895099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:27:02.896129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:27:02.896129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:27:02.896129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:27:02.896129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:27:02.896129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:27:02.896129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:27:02.896129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:27:02.896129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:27:02.896129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 618/1244] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:27:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:27:02.901337 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe1807-bcd7-43a3-9bde-8bc46fdc61cb None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:03.000268 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe1807-bcd7-43a3-9bde-8bc46fdc61cb None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:03.031891 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384bbe8-ab3f-4020-8e42-4e6a252c5d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:03.032480 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384bbe8-ab3f-4020-8e42-4e6a252c5d1f None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:03.032754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384bbe8-ab3f-4020-8e42-4e6a252c5d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:03.033019 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384bbe8-ab3f-4020-8e42-4e6a252c5d1f None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:03.033717 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7384bbe8-ab3f-4020-8e42-4e6a252c5d1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:27:03.034015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7384bbe8-ab3f-4020-8e42-4e6a252c5d1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:03.036054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7384bbe8-ab3f-4020-8e42-4e6a252c5d1f None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:03.047392 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 627/1245] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:27:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:27:03.054369 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e81163-80f7-4c78-9b9b-d50ec0ba32eb None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:03.153535 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e81163-80f7-4c78-9b9b-d50ec0ba32eb None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:03.187649 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79c059-6275-4be3-a768-a515a792e6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:03.188254 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79c059-6275-4be3-a768-a515a792e6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:03.188528 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79c059-6275-4be3-a768-a515a792e6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:03.188797 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79c059-6275-4be3-a768-a515a792e6c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:03.189553 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79c059-6275-4be3-a768-a515a792e6c7 None admin] RBAC: Adding query filter params (role.id=402c3e6fe3364285bd72784a297db11e, scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4, user.id=98c23a782b9b4144bace1248c3b2e754) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:27:03.189856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79c059-6275-4be3-a768-a515a792e6c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:03.191313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79c059-6275-4be3-a768-a515a792e6c7 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:03.196975 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 619/1246] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:27:03 2022] GET /identity/v3/role_assignments?user.id=98c23a782b9b4144bace1248c3b2e754&scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4&role.id=402c3e6fe3364285bd72784a297db11e => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:27:05.220784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3992435-e9a5-4221-8d3f-8e6852e1a17a None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:05.221369 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3992435-e9a5-4221-8d3f-8e6852e1a17a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:05.221369 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3992435-e9a5-4221-8d3f-8e6852e1a17a None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:05.223188 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 628/1247] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:27:05.228201 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72576404-0b0b-4bba-8ec0-e9d270a7d595 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:05.228620 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72576404-0b0b-4bba-8ec0-e9d270a7d595 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:05.228749 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72576404-0b0b-4bba-8ec0-e9d270a7d595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:05.235007 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:05.247346 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-72576404-0b0b-4bba-8ec0-e9d270a7d595 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:27:05.248565 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-72576404-0b0b-4bba-8ec0-e9d270a7d595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:27:05.314414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 620/1248] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:27:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:27:05.336774 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3992435-e9a5-4221-8d3f-8e6852e1a17a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:05.355134 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:05.438384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3992435-e9a5-4221-8d3f-8e6852e1a17a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:05.475254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9fa1a-264d-4f84-ad4e-ab4473ca15b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:05.475254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9fa1a-264d-4f84-ad4e-ab4473ca15b1 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:05.475254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9fa1a-264d-4f84-ad4e-ab4473ca15b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:05.475254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9fa1a-264d-4f84-ad4e-ab4473ca15b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:05.483416 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e9fa1a-264d-4f84-ad4e-ab4473ca15b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:27:05.488756 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-f8e9fa1a-264d-4f84-ad4e-ab4473ca15b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:27:05.488756 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:27:05.488756 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:27:05.488756 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:27:05.488756 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:27:05.488756 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:27:05.488756 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:27:05.488756 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:27:05.488756 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:27:05.488756 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:27:05.488756 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:27:05.488756 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:27:05.488756 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:27:05.488756 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:27:05.488756 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:27:05.488756 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:27:05.489865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:27:05.489865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:27:05.489865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:27:05.489865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:27:05.489865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:27:05.489865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:27:05.489865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:27:05.489865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:27:05.489865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:27:05.489865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:27:05.489865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:27:05.489865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:27:05.489865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:27:05.489865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:27:05.489865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:27:05.489865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:27:05.490745 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:27:05.490745 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:27:05.490745 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:27:05.490745 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:27:05.490745 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:27:05.490745 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:27:05.490745 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:27:05.490745 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:27:05.490745 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:27:05.490745 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:27:05.490745 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:27:05.490745 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:27:05.490745 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 629/1249] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:27:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:27:05.495900 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72576404-0b0b-4bba-8ec0-e9d270a7d595 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:05.591830 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72576404-0b0b-4bba-8ec0-e9d270a7d595 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:05.620085 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:05.624687 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e3fef-b1cb-4a60-bd71-0b6b4f36b6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:05.625651 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e3fef-b1cb-4a60-bd71-0b6b4f36b6eb None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:05.625934 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e3fef-b1cb-4a60-bd71-0b6b4f36b6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:05.626206 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e3fef-b1cb-4a60-bd71-0b6b4f36b6eb None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:05.626872 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e3fef-b1cb-4a60-bd71-0b6b4f36b6eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:27:05.627167 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e3fef-b1cb-4a60-bd71-0b6b4f36b6eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:05.628590 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e3fef-b1cb-4a60-bd71-0b6b4f36b6eb None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:05.639749 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 621/1250] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:27:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:27:05.646062 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9fa1a-264d-4f84-ad4e-ab4473ca15b1 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:05.723130 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:05.744590 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9fa1a-264d-4f84-ad4e-ab4473ca15b1 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:05.777010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8641dee-969b-4fab-bca5-1a537c9e344d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:05.777683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8641dee-969b-4fab-bca5-1a537c9e344d None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:05.777960 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8641dee-969b-4fab-bca5-1a537c9e344d None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:05.778227 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8641dee-969b-4fab-bca5-1a537c9e344d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:05.789033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8641dee-969b-4fab-bca5-1a537c9e344d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:05.790499 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8641dee-969b-4fab-bca5-1a537c9e344d None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:05.804164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-f8641dee-969b-4fab-bca5-1a537c9e344d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:27:05.804164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:27:05.804164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:27:05.804164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:27:05.804164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:27:05.804164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:27:05.804164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:27:05.804164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:27:05.804164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:27:05.804164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:27:05.804164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:27:05.804164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:27:05.804164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:27:05.804164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:27:05.804164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:27:05.804164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:27:05.805271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:27:05.805271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:27:05.805271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:27:05.805271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:27:05.805271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:27:05.805271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:27:05.805271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:27:05.805271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:27:05.805271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:27:05.805271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:27:05.805271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:27:05.805271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:27:05.805271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:27:05.805271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:27:05.805271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:27:05.805271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:27:05.805271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:27:05.806394 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:27:05.806394 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:27:05.806394 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:27:05.806394 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 630/1251] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:27:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:27:05.811101 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e3fef-b1cb-4a60-bd71-0b6b4f36b6eb None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:05.911285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e3fef-b1cb-4a60-bd71-0b6b4f36b6eb None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:05.944041 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cffd7-28ad-4216-8948-71bb946c3479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:05.944636 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cffd7-28ad-4216-8948-71bb946c3479 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:05.944917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cffd7-28ad-4216-8948-71bb946c3479 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:05.945222 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cffd7-28ad-4216-8948-71bb946c3479 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:05.945891 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558cffd7-28ad-4216-8948-71bb946c3479 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:27:05.946182 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558cffd7-28ad-4216-8948-71bb946c3479 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:05.947526 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558cffd7-28ad-4216-8948-71bb946c3479 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:05.959556 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 622/1252] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:27:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:27:05.965378 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8641dee-969b-4fab-bca5-1a537c9e344d None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:06.064342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8641dee-969b-4fab-bca5-1a537c9e344d None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:06.090772 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:06.098551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41f213-cb20-46e5-a68a-dd0335c6aa9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:06.099177 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41f213-cb20-46e5-a68a-dd0335c6aa9b None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:06.099452 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41f213-cb20-46e5-a68a-dd0335c6aa9b None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:06.099832 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41f213-cb20-46e5-a68a-dd0335c6aa9b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:06.106933 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d41f213-cb20-46e5-a68a-dd0335c6aa9b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:06.108370 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d41f213-cb20-46e5-a68a-dd0335c6aa9b None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:06.119242 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-0d41f213-cb20-46e5-a68a-dd0335c6aa9b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 08:27:06.119242 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:27:06.119242 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:27:06.119242 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:27:06.119242 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:27:06.119242 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:27:06.119242 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:27:06.119242 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:27:06.119242 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:27:06.119242 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:27:06.119242 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:27:06.119242 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:27:06.119242 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:27:06.119242 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:27:06.119242 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:27:06.119242 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:27:06.123356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:27:06.123356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:27:06.123356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:27:06.123356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:27:06.123356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:27:06.123356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:27:06.123356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:27:06.123356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:27:06.123356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:27:06.123356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:27:06.123356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:27:06.123356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:27:06.123356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:27:06.123356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:27:06.123356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:27:06.123356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:27:06.123356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:27:06.123356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:27:06.124497 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:27:06.124497 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:27:06.124497 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:27:06.124497 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:27:06.124497 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:27:06.124497 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:27:06.124497 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:27:06.124497 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:27:06.124497 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 08:27:06.124497 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:27:06.124497 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 631/1253] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:27:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:27:06.126320 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cffd7-28ad-4216-8948-71bb946c3479 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:06.180816 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:06.224049 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cffd7-28ad-4216-8948-71bb946c3479 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:06.258035 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2260a2-38bb-43a6-90bc-88a027482700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:06.258635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2260a2-38bb-43a6-90bc-88a027482700 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:06.258913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2260a2-38bb-43a6-90bc-88a027482700 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:06.259182 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2260a2-38bb-43a6-90bc-88a027482700 None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:06.259899 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2260a2-38bb-43a6-90bc-88a027482700 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:27:06.260202 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2260a2-38bb-43a6-90bc-88a027482700 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:06.261652 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2260a2-38bb-43a6-90bc-88a027482700 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:06.279371 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 623/1254] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Aug 8 08:27:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:27:06.285796 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41f213-cb20-46e5-a68a-dd0335c6aa9b None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:06.385980 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41f213-cb20-46e5-a68a-dd0335c6aa9b None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:06.420426 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f052ab1-848f-425c-98ac-b7b94c27e464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:06.421044 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f052ab1-848f-425c-98ac-b7b94c27e464 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:06.421352 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f052ab1-848f-425c-98ac-b7b94c27e464 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:06.421626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f052ab1-848f-425c-98ac-b7b94c27e464 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:06.479014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:06.491607 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f052ab1-848f-425c-98ac-b7b94c27e464 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:06.493059 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f052ab1-848f-425c-98ac-b7b94c27e464 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:06.591541 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 632/1255] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:27:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 08:27:06.597444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2260a2-38bb-43a6-90bc-88a027482700 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:06.698617 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2260a2-38bb-43a6-90bc-88a027482700 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:06.711409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:06.731676 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0221c4-5636-4f17-b534-6ea000650eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:06.732294 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0221c4-5636-4f17-b534-6ea000650eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:06.732570 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0221c4-5636-4f17-b534-6ea000650eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:06.732838 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0221c4-5636-4f17-b534-6ea000650eb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:06.742621 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0221c4-5636-4f17-b534-6ea000650eb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:06.744054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0221c4-5636-4f17-b534-6ea000650eb3 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:06.758452 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-ca0221c4-5636-4f17-b534-6ea000650eb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:27:06.758452 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:27:06.758452 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:27:06.758452 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:27:06.758452 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:27:06.758452 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:27:06.758452 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:27:06.758452 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:27:06.758452 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:27:06.758452 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:27:06.758452 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:27:06.758452 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:27:06.758452 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:27:06.758452 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:27:06.758452 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:27:06.758452 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:27:06.764125 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:27:06.764125 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:27:06.764125 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:27:06.764125 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:27:06.764125 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:27:06.764125 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:27:06.764125 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:27:06.764125 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:27:06.764125 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:27:06.764125 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:27:06.764125 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:27:06.764125 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:27:06.764125 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:27:06.764125 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:27:06.764125 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:27:06.764125 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:27:06.764125 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:27:06.765180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:27:06.765180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:27:06.765180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:27:06.765180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 624/1256] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:27:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:27:06.765409 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f052ab1-848f-425c-98ac-b7b94c27e464 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:06.851055 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:06.862306 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f052ab1-848f-425c-98ac-b7b94c27e464 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:06.896581 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30703d-24ea-4b3e-8db3-81048c8f909a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:06.897223 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30703d-24ea-4b3e-8db3-81048c8f909a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:06.897501 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30703d-24ea-4b3e-8db3-81048c8f909a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:06.897774 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30703d-24ea-4b3e-8db3-81048c8f909a None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:06.898416 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30703d-24ea-4b3e-8db3-81048c8f909a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:27:06.898717 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30703d-24ea-4b3e-8db3-81048c8f909a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:06.900088 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30703d-24ea-4b3e-8db3-81048c8f909a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:06.911986 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 633/1257] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:27:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:27:06.917813 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0221c4-5636-4f17-b534-6ea000650eb3 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:07.016631 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0221c4-5636-4f17-b534-6ea000650eb3 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:07.050690 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21ef6c-dfc9-40d0-ae80-3e97ab6d7374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:07.051341 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21ef6c-dfc9-40d0-ae80-3e97ab6d7374 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:07.051617 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21ef6c-dfc9-40d0-ae80-3e97ab6d7374 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:07.051884 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21ef6c-dfc9-40d0-ae80-3e97ab6d7374 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:07.058363 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21ef6c-dfc9-40d0-ae80-3e97ab6d7374 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:07.059834 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21ef6c-dfc9-40d0-ae80-3e97ab6d7374 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:07.070821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-1c21ef6c-dfc9-40d0-ae80-3e97ab6d7374 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:27:07.070821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:27:07.070821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:27:07.070821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:27:07.070821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:27:07.070821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:27:07.070821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:27:07.070821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:27:07.070821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:27:07.070821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:27:07.070821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:27:07.070821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:27:07.070821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:27:07.070821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:27:07.070821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:27:07.071870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:27:07.071870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:27:07.071870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:27:07.071870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:27:07.071870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:27:07.071870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:27:07.071870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:27:07.071870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:27:07.071870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:27:07.071870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:27:07.071870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:27:07.071870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:27:07.071870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:27:07.071870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:27:07.071870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:27:07.071870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:27:07.071870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:27:07.072948 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:27:07.072948 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:27:07.072948 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:27:07.072948 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:27:07.072948 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:27:07.072948 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:27:07.072948 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:27:07.072948 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:27:07.072948 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 625/1258] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:27:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:27:07.078081 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30703d-24ea-4b3e-8db3-81048c8f909a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:07.176804 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30703d-24ea-4b3e-8db3-81048c8f909a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:07.209223 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41bb69-657e-4450-84f3-9b3bcf7fd2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:07.209816 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41bb69-657e-4450-84f3-9b3bcf7fd2cf None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:07.210092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41bb69-657e-4450-84f3-9b3bcf7fd2cf None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:07.210361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41bb69-657e-4450-84f3-9b3bcf7fd2cf None admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:07.211036 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41bb69-657e-4450-84f3-9b3bcf7fd2cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:27:07.211333 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41bb69-657e-4450-84f3-9b3bcf7fd2cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:07.212790 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41bb69-657e-4450-84f3-9b3bcf7fd2cf None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:07.225254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 634/1259] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:27:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:27:07.231473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21ef6c-dfc9-40d0-ae80-3e97ab6d7374 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:07.257035 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:07.329814 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21ef6c-dfc9-40d0-ae80-3e97ab6d7374 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:07.363709 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7a930-a31b-4219-82f2-91258818350d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:07.364391 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7a930-a31b-4219-82f2-91258818350d None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:07.364667 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7a930-a31b-4219-82f2-91258818350d None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:07.364938 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7a930-a31b-4219-82f2-91258818350d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:07.365697 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e7a930-a31b-4219-82f2-91258818350d None admin] RBAC: Adding query filter params (role.id=c1e514a4b5424f959b069f36cbfd8147, scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4, user.id=98c23a782b9b4144bace1248c3b2e754) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:27:07.366003 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e7a930-a31b-4219-82f2-91258818350d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:07.367410 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e7a930-a31b-4219-82f2-91258818350d None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:07.372586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 626/1260] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:27:07 2022] GET /identity/v3/role_assignments?user.id=98c23a782b9b4144bace1248c3b2e754&scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4&role.id=c1e514a4b5424f959b069f36cbfd8147 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:27:09.370698 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a2006-e124-4f6e-ac22-7a17984f9a96 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:09.371231 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a2006-e124-4f6e-ac22-7a17984f9a96 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:09.371231 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a2006-e124-4f6e-ac22-7a17984f9a96 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:09.372552 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 635/1261] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:27:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:27:09.377685 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee5725-0156-4c2c-ba1d-33a8010b898a None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:09.378157 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee5725-0156-4c2c-ba1d-33a8010b898a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:09.378866 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee5725-0156-4c2c-ba1d-33a8010b898a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:09.396944 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-51ee5725-0156-4c2c-ba1d-33a8010b898a None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:27:09.398157 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-51ee5725-0156-4c2c-ba1d-33a8010b898a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:27:09.463922 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 627/1262] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:27:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:27:09.486179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a2006-e124-4f6e-ac22-7a17984f9a96 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:09.498038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:09.585948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a2006-e124-4f6e-ac22-7a17984f9a96 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:09.619014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2cb6e-b93c-45c6-b2c1-9c32f6ffed76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:09.619658 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2cb6e-b93c-45c6-b2c1-9c32f6ffed76 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:09.619969 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2cb6e-b93c-45c6-b2c1-9c32f6ffed76 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:09.620238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2cb6e-b93c-45c6-b2c1-9c32f6ffed76 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:09.630834 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2cb6e-b93c-45c6-b2c1-9c32f6ffed76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:27:09.635963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-ebe2cb6e-b93c-45c6-b2c1-9c32f6ffed76 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:27:09.635963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:27:09.635963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:27:09.635963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:27:09.635963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:27:09.635963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:27:09.635963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:27:09.635963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:27:09.635963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:27:09.635963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:27:09.635963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:27:09.635963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:27:09.635963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:27:09.635963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:27:09.635963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:27:09.635963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:27:09.641360 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:27:09.641360 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:27:09.641360 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:27:09.641360 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:27:09.641360 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:27:09.641360 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:27:09.641360 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:27:09.641360 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:27:09.641360 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:27:09.641360 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:27:09.641360 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:27:09.641360 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:27:09.641360 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:27:09.641360 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:27:09.641360 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:27:09.641360 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:27:09.642235 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:27:09.642235 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:27:09.642235 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:27:09.642235 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:27:09.642235 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:27:09.642235 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:27:09.642235 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:27:09.642235 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:27:09.642235 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:27:09.642235 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:27:09.642235 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:27:09.642235 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:27:09.642235 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 636/1263] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:27:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:27:09.644164 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee5725-0156-4c2c-ba1d-33a8010b898a None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:09.662738 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:09.741282 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee5725-0156-4c2c-ba1d-33a8010b898a None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:09.773850 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76254527-cee3-4c61-884f-06eb51bae066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:09.774674 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76254527-cee3-4c61-884f-06eb51bae066 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:09.775121 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76254527-cee3-4c61-884f-06eb51bae066 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:09.777264 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76254527-cee3-4c61-884f-06eb51bae066 None admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:09.777264 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76254527-cee3-4c61-884f-06eb51bae066 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:27:09.777264 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76254527-cee3-4c61-884f-06eb51bae066 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:09.778561 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76254527-cee3-4c61-884f-06eb51bae066 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:09.787373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 628/1264] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:27:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:27:09.793247 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2cb6e-b93c-45c6-b2c1-9c32f6ffed76 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:09.859089 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:09.892154 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2cb6e-b93c-45c6-b2c1-9c32f6ffed76 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:09.924448 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ed920-7eb0-4db0-b559-7fff254e1f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:09.925879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ed920-7eb0-4db0-b559-7fff254e1f9a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:09.926045 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ed920-7eb0-4db0-b559-7fff254e1f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:09.926247 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ed920-7eb0-4db0-b559-7fff254e1f9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:09.936656 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ed920-7eb0-4db0-b559-7fff254e1f9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:09.938154 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ed920-7eb0-4db0-b559-7fff254e1f9a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:09.952620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-bb9ed920-7eb0-4db0-b559-7fff254e1f9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:27:09.952620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:27:09.952620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:27:09.952620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:27:09.952620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:27:09.952620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:27:09.952620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:27:09.952620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:27:09.952620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:27:09.952620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:27:09.952620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:27:09.952620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:27:09.952620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:27:09.952620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:27:09.952620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:27:09.952620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:27:09.953823 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:27:09.953823 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:27:09.953823 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:27:09.953823 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:27:09.953823 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:27:09.953823 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:27:09.953823 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:27:09.953823 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:27:09.953823 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:27:09.953823 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:27:09.953823 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:27:09.953823 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:27:09.953823 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:27:09.953823 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:27:09.953823 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:27:09.953823 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:27:09.953823 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:27:09.954773 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:27:09.954773 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:27:09.954773 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:27:09.954773 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 637/1265] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:27:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:27:09.959632 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76254527-cee3-4c61-884f-06eb51bae066 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:10.039509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:10.058713 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76254527-cee3-4c61-884f-06eb51bae066 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:10.093169 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4cfbbb-8326-4746-ace4-640fcb523f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:10.093828 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4cfbbb-8326-4746-ace4-640fcb523f48 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:10.094117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4cfbbb-8326-4746-ace4-640fcb523f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:10.094388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4cfbbb-8326-4746-ace4-640fcb523f48 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:10.095038 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4cfbbb-8326-4746-ace4-640fcb523f48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:27:10.095388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4cfbbb-8326-4746-ace4-640fcb523f48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:10.096749 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4cfbbb-8326-4746-ace4-640fcb523f48 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:10.109514 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 629/1266] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:27:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:27:10.115714 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ed920-7eb0-4db0-b559-7fff254e1f9a None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:10.213227 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ed920-7eb0-4db0-b559-7fff254e1f9a None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:10.233850 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:10.247375 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae830f-825f-478c-bd49-cfd27aba4d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:10.248077 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae830f-825f-478c-bd49-cfd27aba4d97 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:10.248356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae830f-825f-478c-bd49-cfd27aba4d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:10.248627 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae830f-825f-478c-bd49-cfd27aba4d97 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:10.258093 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ae830f-825f-478c-bd49-cfd27aba4d97 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:10.259546 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ae830f-825f-478c-bd49-cfd27aba4d97 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:10.271617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-09ae830f-825f-478c-bd49-cfd27aba4d97 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 08:27:10.271617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:27:10.271617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:27:10.271617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:27:10.271617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:27:10.271617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:27:10.271617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:27:10.271617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:27:10.271617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:27:10.271617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:27:10.271617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:27:10.271617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:27:10.271617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:27:10.271617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:27:10.271617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:27:10.271617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:27:10.272762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:27:10.272762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:27:10.272762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:27:10.272762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:27:10.272762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:27:10.272762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:27:10.272762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:27:10.272762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:27:10.272762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:27:10.272762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:27:10.272762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:27:10.272762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:27:10.272762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:27:10.272762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:27:10.272762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:27:10.272762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:27:10.272762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:27:10.272762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:27:10.273987 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:27:10.273987 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:27:10.273987 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:27:10.273987 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:27:10.273987 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:27:10.273987 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:27:10.273987 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:27:10.273987 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:27:10.273987 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 08:27:10.273987 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:27:10.277682 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 638/1267] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:27:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:27:10.278806 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4cfbbb-8326-4746-ace4-640fcb523f48 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:10.377462 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4cfbbb-8326-4746-ace4-640fcb523f48 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:10.414075 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf93135-f88e-4163-aaaf-2c5d075a67eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:10.414677 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf93135-f88e-4163-aaaf-2c5d075a67eb None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:10.414960 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf93135-f88e-4163-aaaf-2c5d075a67eb None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:10.415212 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf93135-f88e-4163-aaaf-2c5d075a67eb None admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:10.416042 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf93135-f88e-4163-aaaf-2c5d075a67eb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:27:10.416224 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf93135-f88e-4163-aaaf-2c5d075a67eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:10.417746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf93135-f88e-4163-aaaf-2c5d075a67eb None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:10.427694 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:10.435941 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 630/1268] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Aug 8 08:27:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:27:10.441718 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae830f-825f-478c-bd49-cfd27aba4d97 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:10.541778 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae830f-825f-478c-bd49-cfd27aba4d97 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:10.575402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a2fa2-357f-4ec8-a609-33875bd42abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:10.576017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a2fa2-357f-4ec8-a609-33875bd42abc None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:10.576292 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a2fa2-357f-4ec8-a609-33875bd42abc None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:10.576556 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a2fa2-357f-4ec8-a609-33875bd42abc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:10.587329 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a2fa2-357f-4ec8-a609-33875bd42abc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:10.588754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a2fa2-357f-4ec8-a609-33875bd42abc None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:10.602995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-3a5a2fa2-357f-4ec8-a609-33875bd42abc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:27:10.602995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:27:10.602995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:27:10.602995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:27:10.602995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:27:10.602995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:27:10.602995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:27:10.602995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:27:10.602995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:27:10.602995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:27:10.602995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:27:10.602995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:27:10.602995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:27:10.602995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:27:10.602995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:27:10.602995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:27:10.609526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:27:10.609526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:27:10.609526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:27:10.609526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:27:10.609526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:27:10.609526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:27:10.609526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:27:10.609526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:27:10.609526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:27:10.609526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:27:10.609526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:27:10.609526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:27:10.609526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:27:10.609526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:27:10.609526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:27:10.609526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:27:10.609526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:27:10.610653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:27:10.610653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:27:10.610653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:27:10.610653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 639/1269] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:27:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:27:10.610928 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf93135-f88e-4163-aaaf-2c5d075a67eb None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:10.708583 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf93135-f88e-4163-aaaf-2c5d075a67eb None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:10.743015 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50793f5d-db23-46fa-ab51-1f4b32cd1b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:10.743608 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50793f5d-db23-46fa-ab51-1f4b32cd1b99 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:10.743884 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50793f5d-db23-46fa-ab51-1f4b32cd1b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:10.744152 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50793f5d-db23-46fa-ab51-1f4b32cd1b99 None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:10.744809 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50793f5d-db23-46fa-ab51-1f4b32cd1b99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:27:10.745105 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50793f5d-db23-46fa-ab51-1f4b32cd1b99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:10.746538 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50793f5d-db23-46fa-ab51-1f4b32cd1b99 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:10.757917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 631/1270] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:27:10.763631 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a2fa2-357f-4ec8-a609-33875bd42abc None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:10.796146 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:10.863970 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a2fa2-357f-4ec8-a609-33875bd42abc None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:10.896551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c416722f-9b3f-4cc8-ba16-2a7f5a7661e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:10.897246 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416722f-9b3f-4cc8-ba16-2a7f5a7661e5 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:10.897529 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416722f-9b3f-4cc8-ba16-2a7f5a7661e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:10.897797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c416722f-9b3f-4cc8-ba16-2a7f5a7661e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:10.904164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c416722f-9b3f-4cc8-ba16-2a7f5a7661e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:10.905655 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c416722f-9b3f-4cc8-ba16-2a7f5a7661e5 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:10.917165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-c416722f-9b3f-4cc8-ba16-2a7f5a7661e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:27:10.917165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:27:10.917165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:27:10.917165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:27:10.917165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:27:10.917165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:27:10.917165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:27:10.917165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:27:10.917165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:27:10.917165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:27:10.917165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:27:10.917165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:27:10.917165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:27:10.917165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:27:10.917165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:27:10.923123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:27:10.923123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:27:10.923123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:27:10.923123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:27:10.923123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:27:10.923123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:27:10.923123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:27:10.923123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:27:10.923123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:27:10.923123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:27:10.923123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:27:10.923123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:27:10.923123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:27:10.923123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:27:10.923123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:27:10.923123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:27:10.923123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:27:10.924092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:27:10.924092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:27:10.924092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:27:10.924092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:27:10.924092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:27:10.924092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:27:10.924092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:27:10.924092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:27:10.924092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 640/1271] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:27:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:27:10.924582 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50793f5d-db23-46fa-ab51-1f4b32cd1b99 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:10.983209 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:11.023831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50793f5d-db23-46fa-ab51-1f4b32cd1b99 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:11.056831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c274d878-3671-4dbe-ad09-b2bda3f7c9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:11.057389 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274d878-3671-4dbe-ad09-b2bda3f7c9b9 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:11.057673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274d878-3671-4dbe-ad09-b2bda3f7c9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:11.057942 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274d878-3671-4dbe-ad09-b2bda3f7c9b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:11.058624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c274d878-3671-4dbe-ad09-b2bda3f7c9b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:27:11.058928 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c274d878-3671-4dbe-ad09-b2bda3f7c9b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:11.060401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c274d878-3671-4dbe-ad09-b2bda3f7c9b9 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:11.072700 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 632/1272] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:27:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:27:11.078806 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c416722f-9b3f-4cc8-ba16-2a7f5a7661e5 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:11.177165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:11.185615 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c416722f-9b3f-4cc8-ba16-2a7f5a7661e5 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:11.217886 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b977f-27ed-4481-9b06-2b7c80b99eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:11.218536 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b977f-27ed-4481-9b06-2b7c80b99eb3 None admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:11.218816 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b977f-27ed-4481-9b06-2b7c80b99eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:11.219089 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b977f-27ed-4481-9b06-2b7c80b99eb3 None admin] PATH_INFO: `/v3/projects/71e5e82c9f454cdd91dfeac5a57968c4/users/98c23a782b9b4144bace1248c3b2e754/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:11.220975 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b977f-27ed-4481-9b06-2b7c80b99eb3 None admin] RBAC: Authorizing `identity:create_grant(project_id=71e5e82c9f454cdd91dfeac5a57968c4, user_id=98c23a782b9b4144bace1248c3b2e754, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:11.223056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b977f-27ed-4481-9b06-2b7c80b99eb3 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:11.290806 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 641/1273] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 08:27:11 2022] PUT /identity/v3/projects/71e5e82c9f454cdd91dfeac5a57968c4/users/98c23a782b9b4144bace1248c3b2e754/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:27:13.252424 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43cd3b9-eebd-40be-81b7-50ab369fb88e None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:13.253077 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43cd3b9-eebd-40be-81b7-50ab369fb88e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:13.253077 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43cd3b9-eebd-40be-81b7-50ab369fb88e None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:13.254585 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 633/1274] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:27:13.260053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e313232-b813-4952-a48c-537c3ba2c634 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:13.260485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e313232-b813-4952-a48c-537c3ba2c634 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:13.260582 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e313232-b813-4952-a48c-537c3ba2c634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:13.279014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-3e313232-b813-4952-a48c-537c3ba2c634 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:27:13.280186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3e313232-b813-4952-a48c-537c3ba2c634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:27:13.348693 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 642/1275] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:27:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:27:13.372478 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43cd3b9-eebd-40be-81b7-50ab369fb88e None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:13.474356 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43cd3b9-eebd-40be-81b7-50ab369fb88e None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:13.490647 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:13.506814 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c9407-2371-4956-a78f-4e2ee30de62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:13.507374 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c9407-2371-4956-a78f-4e2ee30de62b None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:13.507649 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c9407-2371-4956-a78f-4e2ee30de62b None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:13.507916 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c9407-2371-4956-a78f-4e2ee30de62b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:13.517862 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c9407-2371-4956-a78f-4e2ee30de62b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 08:27:13.523080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-ff0c9407-2371-4956-a78f-4e2ee30de62b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:27:13.523080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:27:13.523080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:27:13.523080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:27:13.523080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:27:13.523080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:27:13.523080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:27:13.523080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:27:13.523080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:27:13.523080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:27:13.523080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:27:13.523080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:27:13.523080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 08:27:13.523080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 08:27:13.523080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 08:27:13.523080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 08:27:13.524290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 08:27:13.524290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 08:27:13.524290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 08:27:13.524290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 08:27:13.524290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:27:13.524290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:27:13.524290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:27:13.524290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:27:13.524290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:27:13.524290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:27:13.524290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:27:13.524290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:27:13.524290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:27:13.524290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:27:13.524290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:27:13.524290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:27:13.525580 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:27:13.525580 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:27:13.525580 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:27:13.525580 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:27:13.525580 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 08:27:13.525580 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 08:27:13.525580 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 08:27:13.525580 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 08:27:13.525580 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 08:27:13.525580 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 08:27:13.525580 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 08:27:13.525580 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:27:13.525580 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 634/1276] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 08:27:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:27:13.530222 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e313232-b813-4952-a48c-537c3ba2c634 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:13.609967 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:13.627455 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e313232-b813-4952-a48c-537c3ba2c634 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:13.660432 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7dcc5-3af2-4ed8-be2b-ca35efa8e349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:13.661033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7dcc5-3af2-4ed8-be2b-ca35efa8e349 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:13.661337 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7dcc5-3af2-4ed8-be2b-ca35efa8e349 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:13.661607 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7dcc5-3af2-4ed8-be2b-ca35efa8e349 None admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:13.662259 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d7dcc5-3af2-4ed8-be2b-ca35efa8e349 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:27:13.662554 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d7dcc5-3af2-4ed8-be2b-ca35efa8e349 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:13.663911 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d7dcc5-3af2-4ed8-be2b-ca35efa8e349 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:13.671649 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 643/1277] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 08:27:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:27:13.677973 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c9407-2371-4956-a78f-4e2ee30de62b None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:13.780882 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c9407-2371-4956-a78f-4e2ee30de62b None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:13.813616 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff91884-2ddf-4bb1-93ab-227ebdd65b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:13.814235 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff91884-2ddf-4bb1-93ab-227ebdd65b56 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:13.814512 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff91884-2ddf-4bb1-93ab-227ebdd65b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:13.814781 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff91884-2ddf-4bb1-93ab-227ebdd65b56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:13.825487 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff91884-2ddf-4bb1-93ab-227ebdd65b56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:13.826959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff91884-2ddf-4bb1-93ab-227ebdd65b56 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:13.841868 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-8ff91884-2ddf-4bb1-93ab-227ebdd65b56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:27:13.841868 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:27:13.841868 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:27:13.841868 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:27:13.841868 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:27:13.841868 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:27:13.841868 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:27:13.841868 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:27:13.841868 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:27:13.841868 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:27:13.841868 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:27:13.841868 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:27:13.841868 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:27:13.841868 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:27:13.841868 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:27:13.841868 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:27:13.842952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:27:13.842952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:27:13.842952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:27:13.842952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:27:13.842952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:27:13.842952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:27:13.842952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:27:13.842952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:27:13.842952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:27:13.842952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:27:13.842952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:27:13.842952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:27:13.842952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:27:13.842952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:27:13.842952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:27:13.842952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:27:13.842952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:27:13.843972 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:27:13.843972 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:27:13.843972 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:27:13.843972 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 635/1278] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:27:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:27:13.849977 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7dcc5-3af2-4ed8-be2b-ca35efa8e349 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:13.948156 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7dcc5-3af2-4ed8-be2b-ca35efa8e349 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:13.982483 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070604c1-92d2-4cc4-93ae-54c3ce591ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:13.983077 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070604c1-92d2-4cc4-93ae-54c3ce591ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:13.983372 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070604c1-92d2-4cc4-93ae-54c3ce591ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:13.983621 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070604c1-92d2-4cc4-93ae-54c3ce591ba2 None admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:13.984249 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070604c1-92d2-4cc4-93ae-54c3ce591ba2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:27:13.984545 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070604c1-92d2-4cc4-93ae-54c3ce591ba2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:13.985981 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070604c1-92d2-4cc4-93ae-54c3ce591ba2 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:13.997597 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 644/1279] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:27:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:27:14.003624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff91884-2ddf-4bb1-93ab-227ebdd65b56 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:14.034275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:14.101802 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff91884-2ddf-4bb1-93ab-227ebdd65b56 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:14.134559 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a8a07-4e7f-46a9-94c7-0b872f1470c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:14.135208 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a8a07-4e7f-46a9-94c7-0b872f1470c7 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:14.135485 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a8a07-4e7f-46a9-94c7-0b872f1470c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:14.135755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a8a07-4e7f-46a9-94c7-0b872f1470c7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:14.142930 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a8a07-4e7f-46a9-94c7-0b872f1470c7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:14.145122 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a8a07-4e7f-46a9-94c7-0b872f1470c7 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:14.155861 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-6e7a8a07-4e7f-46a9-94c7-0b872f1470c7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 08:27:14.155861 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:27:14.155861 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:27:14.155861 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:27:14.155861 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:27:14.155861 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:27:14.155861 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:27:14.155861 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:27:14.155861 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:27:14.155861 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:27:14.155861 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:27:14.155861 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:27:14.155861 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 08:27:14.155861 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 08:27:14.155861 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 08:27:14.155861 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 08:27:14.156991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:27:14.156991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:27:14.156991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 08:27:14.156991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:27:14.156991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 08:27:14.156991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 08:27:14.156991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:27:14.156991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:27:14.156991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:27:14.156991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:27:14.156991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:27:14.156991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:27:14.156991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:27:14.156991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:27:14.156991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:27:14.156991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:27:14.156991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:27:14.156991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:27:14.157990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:27:14.157990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:27:14.157990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 08:27:14.157990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 08:27:14.157990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 08:27:14.157990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 08:27:14.157990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 08:27:14.157990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 08:27:14.157990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 08:27:14.157990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:27:14.157990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 636/1280] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:27:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:27:14.162910 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070604c1-92d2-4cc4-93ae-54c3ce591ba2 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:14.166667 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:14.259054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070604c1-92d2-4cc4-93ae-54c3ce591ba2 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:14.292718 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b742110-42b1-44e2-97a2-a91cd15d5d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:14.293344 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b742110-42b1-44e2-97a2-a91cd15d5d9c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:14.293590 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b742110-42b1-44e2-97a2-a91cd15d5d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:14.293860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b742110-42b1-44e2-97a2-a91cd15d5d9c None admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:14.294579 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b742110-42b1-44e2-97a2-a91cd15d5d9c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:27:14.294880 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b742110-42b1-44e2-97a2-a91cd15d5d9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:14.296298 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b742110-42b1-44e2-97a2-a91cd15d5d9c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:14.312711 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 645/1281] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Aug 8 08:27:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:27:14.319099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a8a07-4e7f-46a9-94c7-0b872f1470c7 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:14.408582 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:14.418738 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a8a07-4e7f-46a9-94c7-0b872f1470c7 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:14.452113 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e07f2-3c84-4037-83b8-1ecc7f5cce63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:14.452711 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e07f2-3c84-4037-83b8-1ecc7f5cce63 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:14.452992 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e07f2-3c84-4037-83b8-1ecc7f5cce63 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:14.453298 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e07f2-3c84-4037-83b8-1ecc7f5cce63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:14.520526 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e07f2-3c84-4037-83b8-1ecc7f5cce63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:14.522001 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e07f2-3c84-4037-83b8-1ecc7f5cce63 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:14.607729 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:14.618557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 637/1282] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 08:27:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 08:27:14.624444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b742110-42b1-44e2-97a2-a91cd15d5d9c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:14.684362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:14.721443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b742110-42b1-44e2-97a2-a91cd15d5d9c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:14.753670 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c85a8a-e960-4500-afa5-3b0c94ab235e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:14.754354 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c85a8a-e960-4500-afa5-3b0c94ab235e None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:14.754636 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c85a8a-e960-4500-afa5-3b0c94ab235e None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:14.754911 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c85a8a-e960-4500-afa5-3b0c94ab235e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:14.765092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c85a8a-e960-4500-afa5-3b0c94ab235e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:14.766566 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c85a8a-e960-4500-afa5-3b0c94ab235e None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:14.781937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-10c85a8a-e960-4500-afa5-3b0c94ab235e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:27:14.781937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:27:14.781937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:27:14.781937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:27:14.781937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:27:14.781937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:27:14.781937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:27:14.781937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:27:14.781937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:27:14.781937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:27:14.781937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:27:14.781937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:27:14.781937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 08:27:14.781937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 08:27:14.781937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 08:27:14.781937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 08:27:14.785304 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:27:14.785304 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:27:14.785304 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:27:14.785304 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:27:14.785304 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:27:14.785304 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:27:14.785304 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:27:14.785304 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:27:14.785304 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:27:14.785304 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:27:14.785304 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:27:14.785304 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:27:14.785304 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:27:14.785304 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:27:14.785304 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:27:14.785304 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:27:14.785304 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 08:27:14.786421 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 08:27:14.786421 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 08:27:14.786421 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:27:14.786421 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 646/1283] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:27:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:27:14.789446 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e07f2-3c84-4037-83b8-1ecc7f5cce63 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:14.891565 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e07f2-3c84-4037-83b8-1ecc7f5cce63 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:14.926223 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c8833-8965-49ad-983f-92fa90d104ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:14.926824 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c8833-8965-49ad-983f-92fa90d104ef None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:14.927126 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c8833-8965-49ad-983f-92fa90d104ef None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:14.927433 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c8833-8965-49ad-983f-92fa90d104ef None admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:14.928068 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c8833-8965-49ad-983f-92fa90d104ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:27:14.928362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c8833-8965-49ad-983f-92fa90d104ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:14.929750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c8833-8965-49ad-983f-92fa90d104ef None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:14.941589 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 638/1284] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 08:27:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:27:14.947441 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c85a8a-e960-4500-afa5-3b0c94ab235e None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:15.047010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c85a8a-e960-4500-afa5-3b0c94ab235e None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:15.060226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:15.079383 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd864df-f692-4cec-909f-b7a120cd7cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:15.079999 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd864df-f692-4cec-909f-b7a120cd7cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:15.080273 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd864df-f692-4cec-909f-b7a120cd7cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:15.080538 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd864df-f692-4cec-909f-b7a120cd7cc8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:15.086921 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd864df-f692-4cec-909f-b7a120cd7cc8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:15.092372 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd864df-f692-4cec-909f-b7a120cd7cc8 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:15.100568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-ffd864df-f692-4cec-909f-b7a120cd7cc8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:27:15.100568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:27:15.100568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:27:15.100568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:27:15.100568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:27:15.100568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:27:15.100568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:27:15.100568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:27:15.100568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:27:15.100568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:27:15.100568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:27:15.100568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:27:15.100568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 08:27:15.100568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 08:27:15.100568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 08:27:15.101653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 08:27:15.101653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:27:15.101653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:27:15.101653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:27:15.101653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:27:15.101653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:27:15.101653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:27:15.101653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:27:15.101653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 08:27:15.101653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:27:15.101653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:27:15.101653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:27:15.101653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:27:15.101653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:27:15.101653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:27:15.101653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:27:15.101653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:27:15.102796 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 08:27:15.102796 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 08:27:15.102796 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 08:27:15.102796 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 08:27:15.102796 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 08:27:15.102796 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 08:27:15.102796 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 08:27:15.102796 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 08:27:15.102796 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 647/1285] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 08:27:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:27:15.107725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c8833-8965-49ad-983f-92fa90d104ef None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:15.156560 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:15.204920 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c8833-8965-49ad-983f-92fa90d104ef None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:15.238598 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec279a1-0280-44ea-af05-d29475a6f984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:15.239235 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec279a1-0280-44ea-af05-d29475a6f984 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:15.239501 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec279a1-0280-44ea-af05-d29475a6f984 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:15.239789 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec279a1-0280-44ea-af05-d29475a6f984 None admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:15.240465 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec279a1-0280-44ea-af05-d29475a6f984 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:27:15.240762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec279a1-0280-44ea-af05-d29475a6f984 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:15.242190 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec279a1-0280-44ea-af05-d29475a6f984 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:15.254195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 639/1286] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 08:27:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:27:15.260426 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd864df-f692-4cec-909f-b7a120cd7cc8 None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:15.367928 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd864df-f692-4cec-909f-b7a120cd7cc8 None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:15.405422 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f39493-0d09-4213-9448-137d1a5b3f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:15.406037 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f39493-0d09-4213-9448-137d1a5b3f73 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:15.406317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f39493-0d09-4213-9448-137d1a5b3f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:15.406639 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f39493-0d09-4213-9448-137d1a5b3f73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:15.407322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f39493-0d09-4213-9448-137d1a5b3f73 None admin] RBAC: Adding query filter params (role.id=c1e514a4b5424f959b069f36cbfd8147, scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4, user.id=98c23a782b9b4144bace1248c3b2e754) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:27:15.407749 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f39493-0d09-4213-9448-137d1a5b3f73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:15.409255 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f39493-0d09-4213-9448-137d1a5b3f73 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:15.423922 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 648/1287] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 08:27:15 2022] GET /identity/v3/role_assignments?user.id=98c23a782b9b4144bace1248c3b2e754&scope.project.id=71e5e82c9f454cdd91dfeac5a57968c4&role.id=c1e514a4b5424f959b069f36cbfd8147 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:27:17.454235 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e525b-ecbe-46ed-8f72-e10c2ce6bd01 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:17.454735 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e525b-ecbe-46ed-8f72-e10c2ce6bd01 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:17.454838 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e525b-ecbe-46ed-8f72-e10c2ce6bd01 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:17.456148 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 640/1288] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:27:17.461295 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d4bf0-1482-48cf-b77b-378cb1fffbd1 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:17.461717 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d4bf0-1482-48cf-b77b-378cb1fffbd1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:17.461848 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d4bf0-1482-48cf-b77b-378cb1fffbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:17.480766 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-735d4bf0-1482-48cf-b77b-378cb1fffbd1 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:27:17.481961 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-735d4bf0-1482-48cf-b77b-378cb1fffbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:27:17.510639 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:17.546348 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 649/1289] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:27:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:27:17.567997 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e525b-ecbe-46ed-8f72-e10c2ce6bd01 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:17.668733 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e525b-ecbe-46ed-8f72-e10c2ce6bd01 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:17.679302 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:17.701512 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca9b2a-5357-4842-b24a-bbd09ed816c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:17.702133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca9b2a-5357-4842-b24a-bbd09ed816c1 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:17.702409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca9b2a-5357-4842-b24a-bbd09ed816c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:17.702679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca9b2a-5357-4842-b24a-bbd09ed816c1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:17.703258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ca9b2a-5357-4842-b24a-bbd09ed816c1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:17.704644 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ca9b2a-5357-4842-b24a-bbd09ed816c1 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:17.713681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-d4ca9b2a-5357-4842-b24a-bbd09ed816c1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 08:27:17.713681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:27:17.713681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 08:27:17.713681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 08:27:17.713681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 08:27:17.713681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 08:27:17.713681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 08:27:17.713681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:27:17.713681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 08:27:17.713681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:27:17.713681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:27:17.713681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:27:17.713681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:27:17.713681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:27:17.713681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:27:17.713681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:27:17.713681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:27:17.714996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:27:17.714996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:27:17.714996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:27:17.714996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:27:17.714996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:27:17.714996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:27:17.714996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 08:27:17.714996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 08:27:17.714996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 08:27:17.714996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 08:27:17.714996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:27:17.714996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:27:17.714996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:27:17.714996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:27:17.714996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:27:17.714996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:27:17.716062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:27:17.716062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:27:17.716062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:27:17.716062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:27:17.716062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:27:17.716062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:27:17.716062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:27:17.716062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:27:17.716062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:27:17.716062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:27:17.716062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 08:27:17.716062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:27:17.716062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 08:27:17.716062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:27:17.716062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 641/1290] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Aug 8 08:27:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:27:17.721137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735d4bf0-1482-48cf-b77b-378cb1fffbd1 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:17.819705 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735d4bf0-1482-48cf-b77b-378cb1fffbd1 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:17.853764 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa578947-675a-496a-80bb-89b433c3207a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:17.854370 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa578947-675a-496a-80bb-89b433c3207a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:17.854645 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa578947-675a-496a-80bb-89b433c3207a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:17.854915 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa578947-675a-496a-80bb-89b433c3207a None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:17.855562 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa578947-675a-496a-80bb-89b433c3207a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:27:17.855859 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa578947-675a-496a-80bb-89b433c3207a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:17.857241 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa578947-675a-496a-80bb-89b433c3207a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:17.861974 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 650/1291] 10.222.0.33 () {56 vars in 1116 bytes} [Mon Aug 8 08:27:17 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:27:17.867941 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca9b2a-5357-4842-b24a-bbd09ed816c1 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:17.965227 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca9b2a-5357-4842-b24a-bbd09ed816c1 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:17.996931 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacbe3e-1900-47e7-ab74-af0052163067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:18.006246 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacbe3e-1900-47e7-ab74-af0052163067 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:18.006246 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacbe3e-1900-47e7-ab74-af0052163067 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:18.006246 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacbe3e-1900-47e7-ab74-af0052163067 None admin] PATH_INFO: `/v3/services` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:18.006246 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bacbe3e-1900-47e7-ab74-af0052163067 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:27:18.006246 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bacbe3e-1900-47e7-ab74-af0052163067 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:18.006246 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bacbe3e-1900-47e7-ab74-af0052163067 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:18.006751 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 642/1292] 10.222.0.33 () {56 vars in 1116 bytes} [Mon Aug 8 08:27:17 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:27:20.014828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8041ae-f54e-49e6-96ea-671afc5590e7 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:20.015384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8041ae-f54e-49e6-96ea-671afc5590e7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:20.015384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8041ae-f54e-49e6-96ea-671afc5590e7 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:20.016697 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 651/1293] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:27:20.021850 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7ba0d-286c-4d11-aed3-81f451231a01 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:20.022260 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7ba0d-286c-4d11-aed3-81f451231a01 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:20.022399 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7ba0d-286c-4d11-aed3-81f451231a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:20.040461 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-8da7ba0d-286c-4d11-aed3-81f451231a01 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:27:20.041657 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8da7ba0d-286c-4d11-aed3-81f451231a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:27:20.071525 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:20.106770 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 643/1294] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:27:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:27:20.130082 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8041ae-f54e-49e6-96ea-671afc5590e7 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:20.160815 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:20.230147 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8041ae-f54e-49e6-96ea-671afc5590e7 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:20.263359 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899216f9-86ba-4853-83f8-240592852e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:20.263957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899216f9-86ba-4853-83f8-240592852e18 None admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:20.264236 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899216f9-86ba-4853-83f8-240592852e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:20.264506 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899216f9-86ba-4853-83f8-240592852e18 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:20.265349 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899216f9-86ba-4853-83f8-240592852e18 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:20.266658 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899216f9-86ba-4853-83f8-240592852e18 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:20.358578 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 652/1295] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Aug 8 08:27:20 2022] POST /identity/v3/services => generated 246 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:27:22.379656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d1e78-f00e-4672-a00f-b069446c111a None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:22.380211 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d1e78-f00e-4672-a00f-b069446c111a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:22.380211 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d1e78-f00e-4672-a00f-b069446c111a None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:22.381602 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 644/1296] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:27:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:27:22.386656 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc230c6-1e75-4030-b393-d245c6e3e37c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:22.393872 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc230c6-1e75-4030-b393-d245c6e3e37c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:22.393872 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc230c6-1e75-4030-b393-d245c6e3e37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:22.405429 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ecc230c6-1e75-4030-b393-d245c6e3e37c None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:27:22.406590 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ecc230c6-1e75-4030-b393-d245c6e3e37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:27:22.481230 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 653/1297] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:27:22 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:27:22.503625 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d1e78-f00e-4672-a00f-b069446c111a None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:22.605744 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d1e78-f00e-4672-a00f-b069446c111a None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:22.639158 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac9c9c-0f1a-433d-bd1d-9bb242601fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:22.639788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac9c9c-0f1a-433d-bd1d-9bb242601fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:22.640064 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac9c9c-0f1a-433d-bd1d-9bb242601fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:22.640329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac9c9c-0f1a-433d-bd1d-9bb242601fe4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:22.640901 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ac9c9c-0f1a-433d-bd1d-9bb242601fe4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:22.642273 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ac9c9c-0f1a-433d-bd1d-9bb242601fe4 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:22.650762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-e2ac9c9c-0f1a-433d-bd1d-9bb242601fe4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 08:27:22.650762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:27:22.650762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 08:27:22.650762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 08:27:22.650762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 08:27:22.650762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 08:27:22.650762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 08:27:22.650762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:27:22.650762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 08:27:22.650762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:27:22.650762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:27:22.650762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:27:22.650762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:27:22.650762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:27:22.650762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:27:22.650762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:27:22.650762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:27:22.651877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:27:22.651877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:27:22.651877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:27:22.651877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:27:22.651877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:27:22.651877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:27:22.651877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 08:27:22.651877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 08:27:22.651877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 08:27:22.651877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 08:27:22.651877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:27:22.651877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:27:22.651877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:27:22.651877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:27:22.651877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:27:22.651877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:27:22.652856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:27:22.652856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:27:22.652856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:27:22.652856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:27:22.652856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:27:22.652856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:27:22.652856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:27:22.652856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:27:22.652856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:27:22.652856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:27:22.652856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 08:27:22.652856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:27:22.652856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 08:27:22.652856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:27:22.652856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 645/1298] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Aug 8 08:27:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 08:27:22.657848 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc230c6-1e75-4030-b393-d245c6e3e37c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:22.677383 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:22.757486 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc230c6-1e75-4030-b393-d245c6e3e37c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:22.791813 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f3a20c-5536-4a7d-a305-4e0f69add60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:22.792409 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3a20c-5536-4a7d-a305-4e0f69add60c None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:22.792683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3a20c-5536-4a7d-a305-4e0f69add60c None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:22.792952 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3a20c-5536-4a7d-a305-4e0f69add60c None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:22.793604 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f3a20c-5536-4a7d-a305-4e0f69add60c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:27:22.793900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f3a20c-5536-4a7d-a305-4e0f69add60c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:22.795290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f3a20c-5536-4a7d-a305-4e0f69add60c None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:22.800447 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 654/1299] 10.222.0.33 () {56 vars in 1116 bytes} [Mon Aug 8 08:27:22 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:27:22.806794 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac9c9c-0f1a-433d-bd1d-9bb242601fe4 None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:22.837854 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:22.904888 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac9c9c-0f1a-433d-bd1d-9bb242601fe4 None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:22.937406 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84362f7e-78ce-4756-8240-76ae3074cab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:22.938001 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84362f7e-78ce-4756-8240-76ae3074cab5 None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:22.938276 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84362f7e-78ce-4756-8240-76ae3074cab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:22.938542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84362f7e-78ce-4756-8240-76ae3074cab5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:22.939225 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84362f7e-78ce-4756-8240-76ae3074cab5 None admin] RBAC: Adding query filter params (interface=public, service_id=ca170c3018e5486a919c0adc4c7acd1c, region_id=RegionOne) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:27:22.939568 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84362f7e-78ce-4756-8240-76ae3074cab5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:22.941718 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84362f7e-78ce-4756-8240-76ae3074cab5 None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:22.946738 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 646/1300] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Aug 8 08:27:22 2022] GET /identity/v3/endpoints?service_id=ca170c3018e5486a919c0adc4c7acd1c&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:27:22.952289 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f3a20c-5536-4a7d-a305-4e0f69add60c None admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:23.030973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:23.048372 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f3a20c-5536-4a7d-a305-4e0f69add60c None admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:23.081250 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dbd294-de35-494d-bcb4-af64e278b5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:23.081938 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbd294-de35-494d-bcb4-af64e278b5b9 None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:23.082123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbd294-de35-494d-bcb4-af64e278b5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:23.082390 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbd294-de35-494d-bcb4-af64e278b5b9 None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:23.082984 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dbd294-de35-494d-bcb4-af64e278b5b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:23.084340 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dbd294-de35-494d-bcb4-af64e278b5b9 None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:23.091925 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 655/1301] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Aug 8 08:27:22 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:27:25.098978 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f023e-adf3-49ff-add0-b1cd4270236e None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:25.099538 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f023e-adf3-49ff-add0-b1cd4270236e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:25.099682 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f023e-adf3-49ff-add0-b1cd4270236e None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:25.101021 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 647/1302] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:27:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:27:25.108099 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f85dc7-3ea5-4d87-871a-49f13a01a0d2 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:25.108527 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f85dc7-3ea5-4d87-871a-49f13a01a0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:25.108647 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f85dc7-3ea5-4d87-871a-49f13a01a0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:25.126770 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-27f85dc7-3ea5-4d87-871a-49f13a01a0d2 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:27:25.127934 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-27f85dc7-3ea5-4d87-871a-49f13a01a0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:27:25.194060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 656/1303] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:27:25 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 08:27:25.216159 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f023e-adf3-49ff-add0-b1cd4270236e None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:25.310232 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:25.317462 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f023e-adf3-49ff-add0-b1cd4270236e None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:25.350586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8d93e-2fa5-4bcc-8620-86cae81be01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:25.351145 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8d93e-2fa5-4bcc-8620-86cae81be01f None admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:25.351425 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8d93e-2fa5-4bcc-8620-86cae81be01f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:25.351695 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8d93e-2fa5-4bcc-8620-86cae81be01f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:25.352301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d8d93e-2fa5-4bcc-8620-86cae81be01f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:25.353683 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d8d93e-2fa5-4bcc-8620-86cae81be01f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:25.362139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-b0d8d93e-2fa5-4bcc-8620-86cae81be01f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 08:27:25.362139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:27:25.362139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 08:27:25.362139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 08:27:25.362139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 08:27:25.362139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 08:27:25.362139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 08:27:25.362139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:27:25.362139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 08:27:25.362139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:27:25.362139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 08:27:25.362139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:27:25.362139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 08:27:25.362139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 08:27:25.362139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 08:27:25.362139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 08:27:25.362139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 08:27:25.363725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 08:27:25.363725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 08:27:25.363725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 08:27:25.363725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 08:27:25.363725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 08:27:25.363725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 08:27:25.363725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 08:27:25.363725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 08:27:25.363725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 08:27:25.363725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 08:27:25.363725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 08:27:25.363725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 08:27:25.363725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 08:27:25.363725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 08:27:25.363725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 08:27:25.363725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 08:27:25.364798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 08:27:25.364798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 08:27:25.364798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 08:27:25.364798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 08:27:25.364798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 08:27:25.364798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 08:27:25.364798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 08:27:25.364798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 08:27:25.364798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 08:27:25.364798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 08:27:25.364798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 08:27:25.364798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 08:27:25.364798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 08:27:25.364798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 08:27:25.364798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 648/1304] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Aug 8 08:27:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 08:27:25.370164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f85dc7-3ea5-4d87-871a-49f13a01a0d2 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:25.445351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:25.467957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f85dc7-3ea5-4d87-871a-49f13a01a0d2 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:25.500696 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04de85-aecf-4e1e-a708-70066a3ee76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:25.501323 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04de85-aecf-4e1e-a708-70066a3ee76a None admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:25.501601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04de85-aecf-4e1e-a708-70066a3ee76a None admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:25.501876 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04de85-aecf-4e1e-a708-70066a3ee76a None admin] PATH_INFO: `/v3/services` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:25.502494 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04de85-aecf-4e1e-a708-70066a3ee76a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:27:25.502790 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04de85-aecf-4e1e-a708-70066a3ee76a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:25.504372 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04de85-aecf-4e1e-a708-70066a3ee76a None admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:25.509245 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 657/1305] 10.222.0.33 () {56 vars in 1116 bytes} [Mon Aug 8 08:27:25 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:27:25.514985 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8d93e-2fa5-4bcc-8620-86cae81be01f None admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:27:25.613570 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8d93e-2fa5-4bcc-8620-86cae81be01f None admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:27:25.644759 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889d865-4dab-41f9-a1e9-6ff5e6eb608f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:27:25.645364 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889d865-4dab-41f9-a1e9-6ff5e6eb608f None admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:27:25.645640 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889d865-4dab-41f9-a1e9-6ff5e6eb608f None admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:27:25.645908 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889d865-4dab-41f9-a1e9-6ff5e6eb608f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:27:25.646559 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4889d865-4dab-41f9-a1e9-6ff5e6eb608f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:27:25.647895 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4889d865-4dab-41f9-a1e9-6ff5e6eb608f None admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:27:25.730187 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 649/1306] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:27:25 2022] POST /identity/v3/endpoints => generated 331 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:27:35.546355 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=30 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:27:35.725727 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:33:21.555052 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3904d1-1d63-4d3e-aaa0-1dcffb791017 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:21.556103 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3904d1-1d63-4d3e-aaa0-1dcffb791017 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:21.556609 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3904d1-1d63-4d3e-aaa0-1dcffb791017 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:21.563307 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9200134-485e-43e1-a41f-ee76c7a5e760 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:21.564170 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9200134-485e-43e1-a41f-ee76c7a5e760 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:21.564672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9200134-485e-43e1-a41f-ee76c7a5e760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:21.570430 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 658/1307] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:33:21 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:33:21.663262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-c9200134-485e-43e1-a41f-ee76c7a5e760 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:33:21.664733 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c9200134-485e-43e1-a41f-ee76c7a5e760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:21.823828 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 650/1308] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:33:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:33:21.848526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62714961-0221-4e65-965c-bf01f16fe8f7 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:21.849079 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62714961-0221-4e65-965c-bf01f16fe8f7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:21.849935 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62714961-0221-4e65-965c-bf01f16fe8f7 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:21.852474 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 659/1309] 10.222.0.33 () {54 vars in 965 bytes} [Mon Aug 8 08:33:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:33:21.863485 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68907f-07b8-4928-858a-767e70918d40 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:21.863485 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68907f-07b8-4928-858a-767e70918d40 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:21.863485 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68907f-07b8-4928-858a-767e70918d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:21.983756 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-1b68907f-07b8-4928-858a-767e70918d40 None None] MFA Rules not processed for user `2c3b8b43026c40209bfc42acd2830610`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:33:21.983756 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1b68907f-07b8-4928-858a-767e70918d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:22.010064 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:33:22.095413 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 651/1310] 10.222.0.33 () {58 vars in 1062 bytes} [Mon Aug 8 08:33:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:33:22.105700 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62714961-0221-4e65-965c-bf01f16fe8f7 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:33:22.107636 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-62714961-0221-4e65-965c-bf01f16fe8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:22.220901 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62714961-0221-4e65-965c-bf01f16fe8f7 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:33:22.295800 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5716a7-9367-4392-89e2-05ed6b4eb2ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:33:22.296414 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5716a7-9367-4392-89e2-05ed6b4eb2ed service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:22.296697 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5716a7-9367-4392-89e2-05ed6b4eb2ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:22.296980 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5716a7-9367-4392-89e2-05ed6b4eb2ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:22.298600 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fe5716a7-9367-4392-89e2-05ed6b4eb2ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:22.357663 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:33:22.415239 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5716a7-9367-4392-89e2-05ed6b4eb2ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:33:22.416709 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5716a7-9367-4392-89e2-05ed6b4eb2ed service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:33:22.439834 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 660/1311] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:33:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:33:24.766981 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae2643-a3e3-4a49-b6e6-3526d0af4688 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:24.767509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae2643-a3e3-4a49-b6e6-3526d0af4688 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:24.767509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae2643-a3e3-4a49-b6e6-3526d0af4688 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:24.768788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 652/1312] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:33:24.774583 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c28fd2-7951-40de-9dcd-c98f4cca0807 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:24.775259 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c28fd2-7951-40de-9dcd-c98f4cca0807 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:24.775535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c28fd2-7951-40de-9dcd-c98f4cca0807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:24.854316 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e1c28fd2-7951-40de-9dcd-c98f4cca0807 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:33:24.855535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e1c28fd2-7951-40de-9dcd-c98f4cca0807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:24.958406 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 661/1313] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:33:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:33:24.996399 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae2643-a3e3-4a49-b6e6-3526d0af4688 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:33:24.997957 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-4dae2643-a3e3-4a49-b6e6-3526d0af4688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:25.103699 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae2643-a3e3-4a49-b6e6-3526d0af4688 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:33:25.187912 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:33:25.191212 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306709bc-f580-4b98-bf35-7cbbe1abf5fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:33:25.192528 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306709bc-f580-4b98-bf35-7cbbe1abf5fc service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:25.192777 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306709bc-f580-4b98-bf35-7cbbe1abf5fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:25.193084 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306709bc-f580-4b98-bf35-7cbbe1abf5fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:25.194758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-306709bc-f580-4b98-bf35-7cbbe1abf5fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:25.310154 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306709bc-f580-4b98-bf35-7cbbe1abf5fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:33:25.311599 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306709bc-f580-4b98-bf35-7cbbe1abf5fc service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:33:25.329884 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 653/1314] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:33:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:33:27.857818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464c5e7-8eca-4006-839c-900d14d844a0 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:27.858357 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464c5e7-8eca-4006-839c-900d14d844a0 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:27.858357 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464c5e7-8eca-4006-839c-900d14d844a0 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:27.859684 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 662/1315] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:33:27.864789 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef03a4-14ee-4f39-8648-d2d7bb94dc3d None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:27.865229 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef03a4-14ee-4f39-8648-d2d7bb94dc3d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:27.865363 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef03a4-14ee-4f39-8648-d2d7bb94dc3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:27.928583 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-daef03a4-14ee-4f39-8648-d2d7bb94dc3d None None] MFA Rules not processed for user `21c9b4f481e04483a2d7cd25eb4e9bab`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:33:27.929796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-daef03a4-14ee-4f39-8648-d2d7bb94dc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:27.980724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:33:28.024758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 654/1316] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:33:27 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:33:28.063565 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464c5e7-8eca-4006-839c-900d14d844a0 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:33:28.065159 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7464c5e7-8eca-4006-839c-900d14d844a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:28.075132 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:33:28.167551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464c5e7-8eca-4006-839c-900d14d844a0 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:33:28.239438 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b38347-305f-4354-a37f-d3722c158e7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:33:28.240043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b38347-305f-4354-a37f-d3722c158e7c service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:28.240323 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b38347-305f-4354-a37f-d3722c158e7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:28.240593 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b38347-305f-4354-a37f-d3722c158e7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:28.242207 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-90b38347-305f-4354-a37f-d3722c158e7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:28.304973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:33:28.350396 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b38347-305f-4354-a37f-d3722c158e7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:33:28.351818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b38347-305f-4354-a37f-d3722c158e7c service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:33:28.369394 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 663/1317] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:33:28 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:33:31.201213 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91212848-6637-4e04-a1d4-4d0d87b32792 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:31.201632 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91212848-6637-4e04-a1d4-4d0d87b32792 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:31.201794 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91212848-6637-4e04-a1d4-4d0d87b32792 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:31.203145 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 655/1318] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:33:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:33:31.209318 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d179e-a706-470b-a71c-824ab381c3de None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:31.209773 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d179e-a706-470b-a71c-824ab381c3de None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:31.209872 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d179e-a706-470b-a71c-824ab381c3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:31.273558 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-da9d179e-a706-470b-a71c-824ab381c3de None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:33:31.274737 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-da9d179e-a706-470b-a71c-824ab381c3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:31.368923 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 664/1319] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:33:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:33:31.408061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91212848-6637-4e04-a1d4-4d0d87b32792 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:33:31.409613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-91212848-6637-4e04-a1d4-4d0d87b32792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:31.510328 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91212848-6637-4e04-a1d4-4d0d87b32792 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:33:31.582112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de328828-dc29-4b52-b328-e6ef25ca4aee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:33:31.582785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de328828-dc29-4b52-b328-e6ef25ca4aee service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:31.583066 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de328828-dc29-4b52-b328-e6ef25ca4aee service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:31.583339 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de328828-dc29-4b52-b328-e6ef25ca4aee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:31.585224 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-de328828-dc29-4b52-b328-e6ef25ca4aee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:31.592550 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:33:31.692916 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de328828-dc29-4b52-b328-e6ef25ca4aee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:33:31.694469 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de328828-dc29-4b52-b328-e6ef25ca4aee service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:33:31.713013 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 656/1320] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:33:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:33:34.654967 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52fb88b-60f4-4b67-8d49-172b8840a544 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:34.655523 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52fb88b-60f4-4b67-8d49-172b8840a544 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:34.655523 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52fb88b-60f4-4b67-8d49-172b8840a544 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:34.656826 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 665/1321] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:33:34.662172 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b882a54-5c26-427e-9a39-6c74e5dfd906 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:34.662606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b882a54-5c26-427e-9a39-6c74e5dfd906 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:34.662726 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b882a54-5c26-427e-9a39-6c74e5dfd906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:34.728585 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-8b882a54-5c26-427e-9a39-6c74e5dfd906 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:33:34.729736 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8b882a54-5c26-427e-9a39-6c74e5dfd906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:34.788656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:33:34.826951 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 657/1322] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:33:34.863410 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52fb88b-60f4-4b67-8d49-172b8840a544 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:33:34.864943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c52fb88b-60f4-4b67-8d49-172b8840a544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:34.886629 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:33:34.977724 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52fb88b-60f4-4b67-8d49-172b8840a544 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:33:35.056854 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903f0e6-1f30-4e71-8243-c1ebc0baae31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:33:35.057519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903f0e6-1f30-4e71-8243-c1ebc0baae31 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:35.057801 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903f0e6-1f30-4e71-8243-c1ebc0baae31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:35.058074 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903f0e6-1f30-4e71-8243-c1ebc0baae31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:35.059772 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d903f0e6-1f30-4e71-8243-c1ebc0baae31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:35.134096 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:33:35.173968 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903f0e6-1f30-4e71-8243-c1ebc0baae31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:33:35.175462 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903f0e6-1f30-4e71-8243-c1ebc0baae31 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:33:35.194626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 666/1323] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:33:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:33:37.279511 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0dc856-a531-4922-b86d-e0f4c0ecded9 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:37.280079 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0dc856-a531-4922-b86d-e0f4c0ecded9 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:37.280079 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0dc856-a531-4922-b86d-e0f4c0ecded9 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:37.281467 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 658/1324] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:33:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:33:37.287299 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a050e0-331e-48f3-973d-801a9923a557 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:37.287737 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a050e0-331e-48f3-973d-801a9923a557 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:37.287987 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a050e0-331e-48f3-973d-801a9923a557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:37.352804 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-90a050e0-331e-48f3-973d-801a9923a557 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:33:37.353941 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-90a050e0-331e-48f3-973d-801a9923a557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:37.452371 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 667/1325] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:33:37.489737 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0dc856-a531-4922-b86d-e0f4c0ecded9 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:33:37.491252 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1b0dc856-a531-4922-b86d-e0f4c0ecded9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:37.593650 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0dc856-a531-4922-b86d-e0f4c0ecded9 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:33:37.670852 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5179d36f-5861-4218-a345-548490abbc5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:33:37.671481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179d36f-5861-4218-a345-548490abbc5e service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:37.671769 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179d36f-5861-4218-a345-548490abbc5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:37.672042 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179d36f-5861-4218-a345-548490abbc5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:37.673699 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5179d36f-5861-4218-a345-548490abbc5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:37.690940 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:33:37.790050 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5179d36f-5861-4218-a345-548490abbc5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:33:37.791505 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5179d36f-5861-4218-a345-548490abbc5e service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:33:37.810222 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 659/1326] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:33:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:33:39.879807 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e1534-6a71-4a2c-95b2-8d9121e53c20 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:39.880665 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e1534-6a71-4a2c-95b2-8d9121e53c20 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:39.880944 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e1534-6a71-4a2c-95b2-8d9121e53c20 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:39.882342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 668/1327] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:33:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 08:33:39.887612 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65ae5c-913a-4aad-bb60-f58d62b03114 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:39.887993 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65ae5c-913a-4aad-bb60-f58d62b03114 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:39.888153 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65ae5c-913a-4aad-bb60-f58d62b03114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:39.955165 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-6f65ae5c-913a-4aad-bb60-f58d62b03114 None None] MFA Rules not processed for user `21c9b4f481e04483a2d7cd25eb4e9bab`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:33:39.956942 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6f65ae5c-913a-4aad-bb60-f58d62b03114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:40.020713 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:33:40.053228 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 660/1328] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:33:39 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:33:40.106350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd5df0-8398-4d87-9b46-c084b563dc65 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:40.106635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd5df0-8398-4d87-9b46-c084b563dc65 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:40.106945 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd5df0-8398-4d87-9b46-c084b563dc65 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:40.108289 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 669/1329] 10.222.0.33 () {54 vars in 965 bytes} [Mon Aug 8 08:33:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:33:40.114953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554642de-2f71-4244-823b-21c42f51a035 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:40.115244 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554642de-2f71-4244-823b-21c42f51a035 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:40.115520 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554642de-2f71-4244-823b-21c42f51a035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:40.194345 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-554642de-2f71-4244-823b-21c42f51a035 None None] MFA Rules not processed for user `2c3b8b43026c40209bfc42acd2830610`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:33:40.195531 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-554642de-2f71-4244-823b-21c42f51a035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:40.269723 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 661/1330] 10.222.0.33 () {58 vars in 1062 bytes} [Mon Aug 8 08:33:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:33:40.278758 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd5df0-8398-4d87-9b46-c084b563dc65 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:33:40.280265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-14cd5df0-8398-4d87-9b46-c084b563dc65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:40.311074 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:33:40.386987 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd5df0-8398-4d87-9b46-c084b563dc65 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:33:40.462091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836f7c1-aa00-4298-89d0-d5261601d78c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:33:40.462702 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836f7c1-aa00-4298-89d0-d5261601d78c service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:40.462982 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836f7c1-aa00-4298-89d0-d5261601d78c service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:40.463252 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836f7c1-aa00-4298-89d0-d5261601d78c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:40.464842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d836f7c1-aa00-4298-89d0-d5261601d78c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:40.562751 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:33:40.597746 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836f7c1-aa00-4298-89d0-d5261601d78c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:33:40.599235 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836f7c1-aa00-4298-89d0-d5261601d78c service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:33:40.618596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 670/1331] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:33:40 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:33:43.473627 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6816dff-9178-46f9-bc47-e04a5190708f None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:43.474140 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6816dff-9178-46f9-bc47-e04a5190708f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:43.474140 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6816dff-9178-46f9-bc47-e04a5190708f None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:43.475560 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 662/1332] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:33:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:33:43.480806 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de429ef2-5a86-4331-88ab-a0c15eadea7f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:43.481253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de429ef2-5a86-4331-88ab-a0c15eadea7f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:43.481392 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de429ef2-5a86-4331-88ab-a0c15eadea7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:43.547489 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-de429ef2-5a86-4331-88ab-a0c15eadea7f None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:33:43.548712 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-de429ef2-5a86-4331-88ab-a0c15eadea7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:43.643310 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 671/1333] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:33:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:33:43.679127 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6816dff-9178-46f9-bc47-e04a5190708f None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:33:43.680625 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c6816dff-9178-46f9-bc47-e04a5190708f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:43.749115 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:33:43.783010 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6816dff-9178-46f9-bc47-e04a5190708f None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:33:43.853798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60e96a-6b9f-4635-9223-360df4d9d7ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:33:43.854393 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60e96a-6b9f-4635-9223-360df4d9d7ea service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:43.854668 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60e96a-6b9f-4635-9223-360df4d9d7ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:43.854934 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60e96a-6b9f-4635-9223-360df4d9d7ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:43.856498 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-cf60e96a-6b9f-4635-9223-360df4d9d7ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:43.963979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf60e96a-6b9f-4635-9223-360df4d9d7ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:33:43.965427 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf60e96a-6b9f-4635-9223-360df4d9d7ea service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:33:43.974996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:33:43.982880 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 663/1334] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:33:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:33:45.997093 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbed8e-453c-456b-9132-0ce6d808fcc3 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:45.997667 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbed8e-453c-456b-9132-0ce6d808fcc3 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:45.997667 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbed8e-453c-456b-9132-0ce6d808fcc3 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:45.999025 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 672/1335] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:33:46.004029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d20f47-970f-45a5-8941-a9ea79fa24d1 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:46.004451 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d20f47-970f-45a5-8941-a9ea79fa24d1 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:46.004584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d20f47-970f-45a5-8941-a9ea79fa24d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:46.069874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-06d20f47-970f-45a5-8941-a9ea79fa24d1 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:33:46.071070 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-06d20f47-970f-45a5-8941-a9ea79fa24d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:46.167804 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 664/1336] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:33:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:33:46.208457 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbed8e-453c-456b-9132-0ce6d808fcc3 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:33:46.210246 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c7bbed8e-453c-456b-9132-0ce6d808fcc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:46.249072 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:33:46.311155 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbed8e-453c-456b-9132-0ce6d808fcc3 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:33:46.383637 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d5411-35ae-4822-801a-f3325e33ad9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:33:46.384232 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d5411-35ae-4822-801a-f3325e33ad9b service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:46.384508 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d5411-35ae-4822-801a-f3325e33ad9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:46.384775 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d5411-35ae-4822-801a-f3325e33ad9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:46.386370 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-cb8d5411-35ae-4822-801a-f3325e33ad9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:46.468777 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:33:46.493884 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d5411-35ae-4822-801a-f3325e33ad9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:33:46.495325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d5411-35ae-4822-801a-f3325e33ad9b service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:33:46.514150 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 673/1337] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:33:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:33:49.273354 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4171f-7a16-4943-9b7a-38f6b77fccbb None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:49.273891 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4171f-7a16-4943-9b7a-38f6b77fccbb None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:49.273891 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4171f-7a16-4943-9b7a-38f6b77fccbb None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:49.275245 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 665/1338] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:33:49.280418 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203854bb-c165-438f-9ea3-d2214796ac26 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:49.280837 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203854bb-c165-438f-9ea3-d2214796ac26 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:49.280996 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203854bb-c165-438f-9ea3-d2214796ac26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:49.345900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-203854bb-c165-438f-9ea3-d2214796ac26 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:33:49.347698 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-203854bb-c165-438f-9ea3-d2214796ac26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:49.443697 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 674/1339] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:33:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:33:49.480255 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4171f-7a16-4943-9b7a-38f6b77fccbb None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:33:49.481789 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d7b4171f-7a16-4943-9b7a-38f6b77fccbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:49.555967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:33:49.583717 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4171f-7a16-4943-9b7a-38f6b77fccbb None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:33:49.657632 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4022b8-5fea-42bb-a7c2-baabd65bfb44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:33:49.658236 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4022b8-5fea-42bb-a7c2-baabd65bfb44 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:49.658521 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4022b8-5fea-42bb-a7c2-baabd65bfb44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:49.658791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4022b8-5fea-42bb-a7c2-baabd65bfb44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:49.660622 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-7c4022b8-5fea-42bb-a7c2-baabd65bfb44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:49.766820 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4022b8-5fea-42bb-a7c2-baabd65bfb44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:33:49.768166 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4022b8-5fea-42bb-a7c2-baabd65bfb44 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:33:49.786997 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 666/1340] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:33:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:33:54.069912 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53e0fb-3cee-4a31-ade4-82fe8db1c131 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:54.070427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53e0fb-3cee-4a31-ade4-82fe8db1c131 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:54.070427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53e0fb-3cee-4a31-ade4-82fe8db1c131 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:54.071780 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 675/1341] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:33:54.076880 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b0a48-f28c-4543-955a-71eccfaaa398 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:54.077350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b0a48-f28c-4543-955a-71eccfaaa398 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:54.077513 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b0a48-f28c-4543-955a-71eccfaaa398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:54.083511 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:33:54.155368 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-c57b0a48-f28c-4543-955a-71eccfaaa398 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:33:54.156563 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c57b0a48-f28c-4543-955a-71eccfaaa398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:54.257668 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 667/1342] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:33:54.298932 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53e0fb-3cee-4a31-ade4-82fe8db1c131 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:33:54.300591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1d53e0fb-3cee-4a31-ade4-82fe8db1c131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:54.356644 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:33:54.413862 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53e0fb-3cee-4a31-ade4-82fe8db1c131 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:33:54.494429 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c70fda-489a-4d09-a0da-e728dd2f6f6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:33:54.495178 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c70fda-489a-4d09-a0da-e728dd2f6f6b service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:54.495460 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c70fda-489a-4d09-a0da-e728dd2f6f6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:54.495733 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c70fda-489a-4d09-a0da-e728dd2f6f6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:54.497391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-27c70fda-489a-4d09-a0da-e728dd2f6f6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:54.608019 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:33:54.625473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c70fda-489a-4d09-a0da-e728dd2f6f6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:33:54.626907 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c70fda-489a-4d09-a0da-e728dd2f6f6b service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:33:54.651311 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 676/1343] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:33:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:33:57.632275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137880ec-81a2-4be4-86ca-bc850c06bcab None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:57.632821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137880ec-81a2-4be4-86ca-bc850c06bcab None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:57.632821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137880ec-81a2-4be4-86ca-bc850c06bcab None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:57.634210 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 668/1344] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:33:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:33:57.639554 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966723c2-f29d-45c8-9dde-74dcb11206b8 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:57.639999 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966723c2-f29d-45c8-9dde-74dcb11206b8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:57.640107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966723c2-f29d-45c8-9dde-74dcb11206b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:57.707731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-966723c2-f29d-45c8-9dde-74dcb11206b8 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:33:57.708823 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-966723c2-f29d-45c8-9dde-74dcb11206b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:57.807617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 677/1345] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:33:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:33:57.845850 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137880ec-81a2-4be4-86ca-bc850c06bcab None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:33:57.848270 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-137880ec-81a2-4be4-86ca-bc850c06bcab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:57.933586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:33:57.950199 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137880ec-81a2-4be4-86ca-bc850c06bcab None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:33:58.023683 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da386e-d71c-4df4-9744-4bc64cfea3f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:33:58.024289 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da386e-d71c-4df4-9744-4bc64cfea3f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:33:58.024567 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da386e-d71c-4df4-9744-4bc64cfea3f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:33:58.024837 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da386e-d71c-4df4-9744-4bc64cfea3f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:33:58.026443 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-92da386e-d71c-4df4-9744-4bc64cfea3f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:33:58.139960 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da386e-d71c-4df4-9744-4bc64cfea3f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:33:58.141452 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da386e-d71c-4df4-9744-4bc64cfea3f7 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:33:58.160017 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 669/1346] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:33:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:34:02.352602 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc32623-bbb3-4db0-8b18-c876eb3f5533 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:02.353080 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc32623-bbb3-4db0-8b18-c876eb3f5533 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:02.353214 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc32623-bbb3-4db0-8b18-c876eb3f5533 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:02.354600 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 678/1347] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:34:02.359596 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998028a6-17e8-4648-94ac-a89efec4c245 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:02.360012 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998028a6-17e8-4648-94ac-a89efec4c245 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:02.360147 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998028a6-17e8-4648-94ac-a89efec4c245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:02.378722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:02.438123 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-998028a6-17e8-4648-94ac-a89efec4c245 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:34:02.439378 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-998028a6-17e8-4648-94ac-a89efec4c245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:02.542106 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 670/1348] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:34:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:34:02.580576 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc32623-bbb3-4db0-8b18-c876eb3f5533 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:34:02.582475 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9cc32623-bbb3-4db0-8b18-c876eb3f5533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:02.647886 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:02.701164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc32623-bbb3-4db0-8b18-c876eb3f5533 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:34:02.779501 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e5783-71aa-4374-a6eb-b3df38f74025 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:34:02.780108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e5783-71aa-4374-a6eb-b3df38f74025 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:02.780385 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e5783-71aa-4374-a6eb-b3df38f74025 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:02.780654 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e5783-71aa-4374-a6eb-b3df38f74025 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:02.782268 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1c3e5783-71aa-4374-a6eb-b3df38f74025 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:02.892012 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:02.896302 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e5783-71aa-4374-a6eb-b3df38f74025 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:34:02.902828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e5783-71aa-4374-a6eb-b3df38f74025 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:34:02.916259 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 679/1349] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:34:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:34:07.917172 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c8a2c-0915-4c49-9bf6-d6e01580804d None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:07.917172 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c8a2c-0915-4c49-9bf6-d6e01580804d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:07.917172 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c8a2c-0915-4c49-9bf6-d6e01580804d None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:07.917172 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 671/1350] 10.222.0.33 () {54 vars in 921 bytes} [Mon Aug 8 08:34:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:34:07.921729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffde490-ee45-41ce-9804-f7f938ea98cd None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:07.922473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffde490-ee45-41ce-9804-f7f938ea98cd None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:07.922909 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffde490-ee45-41ce-9804-f7f938ea98cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:08.059872 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4ffde490-ee45-41ce-9804-f7f938ea98cd None None] MFA Rules not processed for user `98c23a782b9b4144bace1248c3b2e754`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:34:08.060970 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4ffde490-ee45-41ce-9804-f7f938ea98cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:08.180343 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 680/1351] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Aug 8 08:34:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:34:12.284676 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b3f24-97ad-43e8-8acf-b56b4618c092 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:12.285247 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b3f24-97ad-43e8-8acf-b56b4618c092 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:12.285247 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b3f24-97ad-43e8-8acf-b56b4618c092 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:12.286792 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 672/1352] 10.222.0.33 () {54 vars in 921 bytes} [Mon Aug 8 08:34:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:34:12.291905 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0afaba8-b330-4b94-a0ec-195731cf64f3 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:12.292334 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0afaba8-b330-4b94-a0ec-195731cf64f3 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:12.292638 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0afaba8-b330-4b94-a0ec-195731cf64f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:12.348109 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:12.372825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e0afaba8-b330-4b94-a0ec-195731cf64f3 None None] MFA Rules not processed for user `98c23a782b9b4144bace1248c3b2e754`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:34:12.374092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e0afaba8-b330-4b94-a0ec-195731cf64f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:12.489341 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 681/1353] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Aug 8 08:34:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:34:12.539938 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=59 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:13.910032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c169a-d82c-4a49-a4f3-bff665a4874b None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:13.910657 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c169a-d82c-4a49-a4f3-bff665a4874b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:13.910657 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c169a-d82c-4a49-a4f3-bff665a4874b None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:13.911916 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 673/1354] 10.222.0.33 () {54 vars in 921 bytes} [Mon Aug 8 08:34:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:34:13.916928 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dcb27-1e3a-40c3-b588-346d17f06991 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:13.917415 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dcb27-1e3a-40c3-b588-346d17f06991 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:13.917558 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dcb27-1e3a-40c3-b588-346d17f06991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:13.995186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-2a2dcb27-1e3a-40c3-b588-346d17f06991 None None] MFA Rules not processed for user `98c23a782b9b4144bace1248c3b2e754`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:34:13.996400 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2a2dcb27-1e3a-40c3-b588-346d17f06991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:14.039265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:14.105529 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 682/1355] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Aug 8 08:34:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:34:17.460085 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf905e1f-3a3f-4667-b0d8-b61168d7e67f None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:17.460957 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf905e1f-3a3f-4667-b0d8-b61168d7e67f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:17.461447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf905e1f-3a3f-4667-b0d8-b61168d7e67f None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:17.463387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 674/1356] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:34:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:34:17.468995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643b9a4-2d3f-4dd4-8b9c-1130f3731b3f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:17.469823 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643b9a4-2d3f-4dd4-8b9c-1130f3731b3f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:17.470282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643b9a4-2d3f-4dd4-8b9c-1130f3731b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:17.542128 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-f643b9a4-2d3f-4dd4-8b9c-1130f3731b3f None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:34:17.542520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f643b9a4-2d3f-4dd4-8b9c-1130f3731b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:17.649370 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 683/1357] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:34:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:34:17.706021 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d0f29-1c3f-4538-8c96-8db8c11141ab None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:17.706632 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d0f29-1c3f-4538-8c96-8db8c11141ab None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:17.707401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d0f29-1c3f-4538-8c96-8db8c11141ab None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:17.709792 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 675/1358] 10.222.0.33 () {54 vars in 962 bytes} [Mon Aug 8 08:34:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:34:17.717420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76841f0-8adb-48ac-84a2-e4a9036aeedd None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:17.718206 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76841f0-8adb-48ac-84a2-e4a9036aeedd None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:17.718965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76841f0-8adb-48ac-84a2-e4a9036aeedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:17.737086 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:17.800146 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c76841f0-8adb-48ac-84a2-e4a9036aeedd None None] MFA Rules not processed for user `a3317f751ec74bde95ec262394001292`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:34:17.801566 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c76841f0-8adb-48ac-84a2-e4a9036aeedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:17.924448 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 684/1359] 10.222.0.33 () {58 vars in 1059 bytes} [Mon Aug 8 08:34:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:34:17.932737 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d0f29-1c3f-4538-8c96-8db8c11141ab None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:34:17.934390 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-4f1d0f29-1c3f-4538-8c96-8db8c11141ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:18.103091 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d0f29-1c3f-4538-8c96-8db8c11141ab None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:34:18.179444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:18.222771 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af51562-1a82-4bd5-9a74-6191ac02188f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:34:18.223177 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af51562-1a82-4bd5-9a74-6191ac02188f service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:18.223457 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af51562-1a82-4bd5-9a74-6191ac02188f service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:18.223802 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af51562-1a82-4bd5-9a74-6191ac02188f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:18.225428 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0af51562-1a82-4bd5-9a74-6191ac02188f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:18.323622 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af51562-1a82-4bd5-9a74-6191ac02188f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:34:18.325217 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af51562-1a82-4bd5-9a74-6191ac02188f service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:34:18.344038 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 676/1360] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:34:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:34:20.600444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2459987-ba4b-4794-801d-3ef2e33694bf None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:20.600980 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2459987-ba4b-4794-801d-3ef2e33694bf None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:20.600980 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2459987-ba4b-4794-801d-3ef2e33694bf None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:20.602403 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 685/1361] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:34:20.607683 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7f544-e12b-46b7-bb83-5b584fe206c9 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:20.608078 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7f544-e12b-46b7-bb83-5b584fe206c9 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:20.608290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7f544-e12b-46b7-bb83-5b584fe206c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:20.681923 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-ddb7f544-e12b-46b7-bb83-5b584fe206c9 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:34:20.683215 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ddb7f544-e12b-46b7-bb83-5b584fe206c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:20.725810 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:20.790648 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 677/1362] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:34:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:34:20.836486 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2459987-ba4b-4794-801d-3ef2e33694bf None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:34:20.837954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b2459987-ba4b-4794-801d-3ef2e33694bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:20.913827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:21.002254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2459987-ba4b-4794-801d-3ef2e33694bf None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:34:21.174902 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a520c-c75a-4489-94de-d3ed6c206814 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:34:21.175292 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a520c-c75a-4489-94de-d3ed6c206814 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:21.175575 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a520c-c75a-4489-94de-d3ed6c206814 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:21.175903 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a520c-c75a-4489-94de-d3ed6c206814 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:21.177584 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-be4a520c-c75a-4489-94de-d3ed6c206814 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:21.202525 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:21.284215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a520c-c75a-4489-94de-d3ed6c206814 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:34:21.286207 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a520c-c75a-4489-94de-d3ed6c206814 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:34:21.305562 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 686/1363] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:34:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:34:23.788932 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e087b-61f1-4a1d-9097-ec79e309d582 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:23.789637 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e087b-61f1-4a1d-9097-ec79e309d582 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:23.789637 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e087b-61f1-4a1d-9097-ec79e309d582 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:23.790904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 678/1364] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:34:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 08:34:23.796211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4401af7-ee8a-4be7-9d6e-c18ec7449b1c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:23.796740 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4401af7-ee8a-4be7-9d6e-c18ec7449b1c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:23.796740 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4401af7-ee8a-4be7-9d6e-c18ec7449b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:23.868716 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d4401af7-ee8a-4be7-9d6e-c18ec7449b1c None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:34:23.869959 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d4401af7-ee8a-4be7-9d6e-c18ec7449b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:23.970694 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:23.973704 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 687/1365] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:34:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:34:24.019405 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba30ac3-61d2-45cb-97f2-9a31a838d15b None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:24.019729 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba30ac3-61d2-45cb-97f2-9a31a838d15b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:24.020306 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba30ac3-61d2-45cb-97f2-9a31a838d15b None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:24.021955 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 679/1366] 10.222.0.33 () {54 vars in 962 bytes} [Mon Aug 8 08:34:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:34:24.027280 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ded30c-6154-4925-86ef-214fa9e9af19 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:24.027632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ded30c-6154-4925-86ef-214fa9e9af19 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:24.027990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ded30c-6154-4925-86ef-214fa9e9af19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:24.108577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-18ded30c-6154-4925-86ef-214fa9e9af19 None None] MFA Rules not processed for user `a3317f751ec74bde95ec262394001292`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:34:24.109878 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-18ded30c-6154-4925-86ef-214fa9e9af19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:24.234924 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 688/1367] 10.222.0.33 () {58 vars in 1059 bytes} [Mon Aug 8 08:34:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:34:24.243311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba30ac3-61d2-45cb-97f2-9a31a838d15b None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:34:24.244856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-fba30ac3-61d2-45cb-97f2-9a31a838d15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:24.405853 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba30ac3-61d2-45cb-97f2-9a31a838d15b None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:34:24.432584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:24.531646 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fab3e2-3e34-422a-83d5-3febfb41a19c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:34:24.532399 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fab3e2-3e34-422a-83d5-3febfb41a19c service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:24.533397 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fab3e2-3e34-422a-83d5-3febfb41a19c service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:24.533624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fab3e2-3e34-422a-83d5-3febfb41a19c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:24.535277 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c9fab3e2-3e34-422a-83d5-3febfb41a19c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:24.636893 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fab3e2-3e34-422a-83d5-3febfb41a19c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:34:24.638762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fab3e2-3e34-422a-83d5-3febfb41a19c service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:34:24.659169 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 680/1368] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:34:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:34:25.604546 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46052d-41c7-4c49-a257-6586be3922b1 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:25.605204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46052d-41c7-4c49-a257-6586be3922b1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:25.605204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46052d-41c7-4c49-a257-6586be3922b1 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:25.606612 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 689/1369] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:34:25.612167 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf503e-3cb6-4db6-9b8f-f410d5857db6 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:25.612639 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf503e-3cb6-4db6-9b8f-f410d5857db6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:25.612737 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf503e-3cb6-4db6-9b8f-f410d5857db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:25.648297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:25.684266 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-51bf503e-3cb6-4db6-9b8f-f410d5857db6 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:34:25.685529 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-51bf503e-3cb6-4db6-9b8f-f410d5857db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:25.791996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 681/1370] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:34:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:34:26.134287 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf1ac7-86d2-4451-9ef2-3525e4d08b19 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:26.134287 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf1ac7-86d2-4451-9ef2-3525e4d08b19 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:26.134287 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf1ac7-86d2-4451-9ef2-3525e4d08b19 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:26.136688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 690/1371] 10.222.0.33 () {54 vars in 964 bytes} [Mon Aug 8 08:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:34:26.140288 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f880ad3-05d2-47c4-9d30-a90e83f4b82f None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:26.140579 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f880ad3-05d2-47c4-9d30-a90e83f4b82f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:26.140857 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f880ad3-05d2-47c4-9d30-a90e83f4b82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:26.218803 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-8f880ad3-05d2-47c4-9d30-a90e83f4b82f None None] MFA Rules not processed for user `3262fa2d9c0d4d8998c9697994b48fef`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:34:26.220532 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8f880ad3-05d2-47c4-9d30-a90e83f4b82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:26.268828 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:26.301444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 682/1372] 10.222.0.33 () {58 vars in 1061 bytes} [Mon Aug 8 08:34:26 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:34:26.309818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf1ac7-86d2-4451-9ef2-3525e4d08b19 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:34:26.311332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fecf1ac7-86d2-4451-9ef2-3525e4d08b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:26.368445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:26.430965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf1ac7-86d2-4451-9ef2-3525e4d08b19 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:34:26.511538 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58fd3a9-b5e3-4c8a-83d9-69fc8e44f82d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:34:26.512321 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58fd3a9-b5e3-4c8a-83d9-69fc8e44f82d service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:26.512859 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58fd3a9-b5e3-4c8a-83d9-69fc8e44f82d service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:26.513275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58fd3a9-b5e3-4c8a-83d9-69fc8e44f82d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:26.515029 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e58fd3a9-b5e3-4c8a-83d9-69fc8e44f82d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:26.625220 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:26.639309 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58fd3a9-b5e3-4c8a-83d9-69fc8e44f82d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:34:26.640972 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58fd3a9-b5e3-4c8a-83d9-69fc8e44f82d service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:34:26.660020 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 691/1373] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:34:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:34:26.903793 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02699cc7-3230-4812-bc3d-cd11abc6ebe3 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:26.903793 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02699cc7-3230-4812-bc3d-cd11abc6ebe3 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:26.903793 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02699cc7-3230-4812-bc3d-cd11abc6ebe3 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:26.903793 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 683/1374] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:34:26.909096 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318f264-6a32-4914-8445-6bc86e8802a3 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:26.909452 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318f264-6a32-4914-8445-6bc86e8802a3 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:26.909613 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318f264-6a32-4914-8445-6bc86e8802a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:26.986920 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d318f264-6a32-4914-8445-6bc86e8802a3 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:34:26.988197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d318f264-6a32-4914-8445-6bc86e8802a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:27.090758 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 692/1375] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:34:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:34:27.135560 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02699cc7-3230-4812-bc3d-cd11abc6ebe3 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:34:27.137008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-02699cc7-3230-4812-bc3d-cd11abc6ebe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:27.284202 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02699cc7-3230-4812-bc3d-cd11abc6ebe3 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:34:27.337589 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:27.400623 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b282629-3167-4785-a37f-a1b3d4223190 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:34:27.401180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b282629-3167-4785-a37f-a1b3d4223190 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:27.401454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b282629-3167-4785-a37f-a1b3d4223190 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:27.401724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b282629-3167-4785-a37f-a1b3d4223190 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:27.403375 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0b282629-3167-4785-a37f-a1b3d4223190 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:27.496433 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b282629-3167-4785-a37f-a1b3d4223190 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:34:27.498052 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b282629-3167-4785-a37f-a1b3d4223190 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:34:27.515653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 684/1376] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:34:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:34:29.694259 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf8b65-dfab-4722-99fd-042af4b1b15b None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:29.694763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf8b65-dfab-4722-99fd-042af4b1b15b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:29.694869 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf8b65-dfab-4722-99fd-042af4b1b15b None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:29.696199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 693/1377] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:34:29.701544 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b4ee6-62d7-47c9-a6a5-218cfbcb7dc4 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:29.701975 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b4ee6-62d7-47c9-a6a5-218cfbcb7dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:29.702080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b4ee6-62d7-47c9-a6a5-218cfbcb7dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:29.781117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-a10b4ee6-62d7-47c9-a6a5-218cfbcb7dc4 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:34:29.782930 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a10b4ee6-62d7-47c9-a6a5-218cfbcb7dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:29.789898 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:29.891241 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 685/1378] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:34:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:34:29.934910 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf8b65-dfab-4722-99fd-042af4b1b15b None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:34:29.936394 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fdaf8b65-dfab-4722-99fd-042af4b1b15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:29.997092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:30.081672 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf8b65-dfab-4722-99fd-042af4b1b15b None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:34:30.235925 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c18d6-5c58-4bef-800d-eda5c8369d3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:34:30.237344 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c18d6-5c58-4bef-800d-eda5c8369d3d service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:30.238077 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c18d6-5c58-4bef-800d-eda5c8369d3d service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:30.238591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c18d6-5c58-4bef-800d-eda5c8369d3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:30.240381 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8e1c18d6-5c58-4bef-800d-eda5c8369d3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:30.263052 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:30.357489 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c18d6-5c58-4bef-800d-eda5c8369d3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:34:30.359144 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c18d6-5c58-4bef-800d-eda5c8369d3d service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:34:30.377467 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 694/1379] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:34:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:34:32.553363 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464d232-00fb-4949-8d12-a21432232223 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:32.553778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464d232-00fb-4949-8d12-a21432232223 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:32.553925 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464d232-00fb-4949-8d12-a21432232223 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:32.555470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 686/1380] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:34:32.560395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0990403-0d9b-4e7f-9c15-05c54cee37dc None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:32.560866 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0990403-0d9b-4e7f-9c15-05c54cee37dc None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:32.560947 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0990403-0d9b-4e7f-9c15-05c54cee37dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:32.636528 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c0990403-0d9b-4e7f-9c15-05c54cee37dc None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:34:32.637820 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c0990403-0d9b-4e7f-9c15-05c54cee37dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:32.737926 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:32.744371 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 695/1381] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:34:32.789043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464d232-00fb-4949-8d12-a21432232223 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:34:32.790619 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2464d232-00fb-4949-8d12-a21432232223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:32.935288 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:32.945950 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464d232-00fb-4949-8d12-a21432232223 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:34:33.064366 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7051ba94-6369-4a9d-898e-7b0172e72246 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:34:33.064874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051ba94-6369-4a9d-898e-7b0172e72246 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:33.065200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051ba94-6369-4a9d-898e-7b0172e72246 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:33.065453 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051ba94-6369-4a9d-898e-7b0172e72246 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:33.067152 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-7051ba94-6369-4a9d-898e-7b0172e72246 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:33.175005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7051ba94-6369-4a9d-898e-7b0172e72246 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:34:33.176651 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7051ba94-6369-4a9d-898e-7b0172e72246 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:34:33.187489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:33.197110 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 687/1382] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:34:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:34:35.571620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287e4ea-0fd8-4fef-ad14-7725c870ac60 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:35.572147 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287e4ea-0fd8-4fef-ad14-7725c870ac60 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:35.572147 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287e4ea-0fd8-4fef-ad14-7725c870ac60 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:35.573818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 696/1383] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 08:34:35.579476 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505c33c-6361-4659-8411-1321b75966e5 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:35.579948 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505c33c-6361-4659-8411-1321b75966e5 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:35.580032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505c33c-6361-4659-8411-1321b75966e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:35.657955 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-3505c33c-6361-4659-8411-1321b75966e5 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:34:35.659305 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-3505c33c-6361-4659-8411-1321b75966e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:35.766749 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 688/1384] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:34:35.808606 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b287e4ea-0fd8-4fef-ad14-7725c870ac60 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:34:35.810183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b287e4ea-0fd8-4fef-ad14-7725c870ac60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:35.979300 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b287e4ea-0fd8-4fef-ad14-7725c870ac60 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:34:36.049552 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:36.100596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe3b7a-686d-440b-a3e7-18888b106892 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:34:36.101232 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe3b7a-686d-440b-a3e7-18888b106892 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:36.101524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe3b7a-686d-440b-a3e7-18888b106892 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:36.101791 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe3b7a-686d-440b-a3e7-18888b106892 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:36.103392 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e0fe3b7a-686d-440b-a3e7-18888b106892 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:36.200035 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe3b7a-686d-440b-a3e7-18888b106892 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:34:36.201689 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe3b7a-686d-440b-a3e7-18888b106892 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:34:36.221584 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 697/1385] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:34:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:34:38.241940 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6f92c-518a-475a-b629-340017649d1f None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:38.242630 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6f92c-518a-475a-b629-340017649d1f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:38.242630 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6f92c-518a-475a-b629-340017649d1f None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:38.243862 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 689/1386] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:34:38.248894 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273923e-37ab-4dbb-9eb9-c3cfeca79485 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:38.249398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273923e-37ab-4dbb-9eb9-c3cfeca79485 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:38.249552 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273923e-37ab-4dbb-9eb9-c3cfeca79485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:38.319107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c273923e-37ab-4dbb-9eb9-c3cfeca79485 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:34:38.320340 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c273923e-37ab-4dbb-9eb9-c3cfeca79485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:38.355273 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:38.423812 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 698/1387] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:34:38.468327 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6f92c-518a-475a-b629-340017649d1f None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:34:38.470240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c1a6f92c-518a-475a-b629-340017649d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:38.554019 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:38.621393 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6f92c-518a-475a-b629-340017649d1f None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:34:38.744168 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec55f7-0782-465d-b461-6e2b0de552dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:34:38.744778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec55f7-0782-465d-b461-6e2b0de552dc service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:38.745060 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec55f7-0782-465d-b461-6e2b0de552dc service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:38.745365 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec55f7-0782-465d-b461-6e2b0de552dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:38.746965 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-cbec55f7-0782-465d-b461-6e2b0de552dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:38.786797 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:38.840777 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec55f7-0782-465d-b461-6e2b0de552dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:34:38.842341 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec55f7-0782-465d-b461-6e2b0de552dc service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:34:38.861165 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 690/1388] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:34:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:34:41.070623 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccd62e-ab4a-4b21-8a1d-28a45c1d1c15 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:41.071194 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccd62e-ab4a-4b21-8a1d-28a45c1d1c15 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:41.071194 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccd62e-ab4a-4b21-8a1d-28a45c1d1c15 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:41.072515 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 699/1389] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:34:41.078019 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99953e99-69c5-416d-98b3-66e18ca386f4 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:41.078451 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99953e99-69c5-416d-98b3-66e18ca386f4 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:41.078567 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99953e99-69c5-416d-98b3-66e18ca386f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:41.148151 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-99953e99-69c5-416d-98b3-66e18ca386f4 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:34:41.149381 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-99953e99-69c5-416d-98b3-66e18ca386f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:41.251544 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 691/1390] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:34:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:34:41.296207 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccd62e-ab4a-4b21-8a1d-28a45c1d1c15 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:34:41.298429 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-70ccd62e-ab4a-4b21-8a1d-28a45c1d1c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:41.445230 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccd62e-ab4a-4b21-8a1d-28a45c1d1c15 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:34:41.462168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:41.558579 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b6ead-3e81-4b37-89aa-54ef033c3e12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:34:41.559212 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b6ead-3e81-4b37-89aa-54ef033c3e12 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:41.559490 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b6ead-3e81-4b37-89aa-54ef033c3e12 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:41.559761 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b6ead-3e81-4b37-89aa-54ef033c3e12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:41.561376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2c9b6ead-3e81-4b37-89aa-54ef033c3e12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:41.653610 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b6ead-3e81-4b37-89aa-54ef033c3e12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:34:41.655222 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b6ead-3e81-4b37-89aa-54ef033c3e12 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:34:41.674075 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 700/1391] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:34:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:34:41.914177 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addfea1b-92a4-4386-9201-1024d0c74ffa None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:41.914469 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addfea1b-92a4-4386-9201-1024d0c74ffa None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:41.914773 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addfea1b-92a4-4386-9201-1024d0c74ffa None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:41.916265 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 692/1392] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:34:41.922150 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849f564-77ab-4001-a99c-58bd849e3571 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:41.922448 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849f564-77ab-4001-a99c-58bd849e3571 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:41.923544 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849f564-77ab-4001-a99c-58bd849e3571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:41.951796 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:41.995951 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c849f564-77ab-4001-a99c-58bd849e3571 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:34:41.997191 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c849f564-77ab-4001-a99c-58bd849e3571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:42.105667 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 701/1393] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:34:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:34:42.161350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1c2f3-0022-498e-b55d-822c661a7b13 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:42.161836 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1c2f3-0022-498e-b55d-822c661a7b13 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:42.161940 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1c2f3-0022-498e-b55d-822c661a7b13 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:42.163218 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 693/1394] 10.222.0.33 () {54 vars in 967 bytes} [Mon Aug 8 08:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:34:42.168139 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1105bc1-0304-4715-8336-05ade3d3d06a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:42.168512 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1105bc1-0304-4715-8336-05ade3d3d06a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:42.168791 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1105bc1-0304-4715-8336-05ade3d3d06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:42.244812 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c1105bc1-0304-4715-8336-05ade3d3d06a None None] MFA Rules not processed for user `627171bf2a2e4a21ba6f800879f8c665`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:34:42.245951 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c1105bc1-0304-4715-8336-05ade3d3d06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:42.287368 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:42.323113 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 702/1395] 10.222.0.33 () {58 vars in 1064 bytes} [Mon Aug 8 08:34:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:34:42.330758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1c2f3-0022-498e-b55d-822c661a7b13 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:34:42.332304 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-90e1c2f3-0022-498e-b55d-822c661a7b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:42.346558 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:42.436849 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1c2f3-0022-498e-b55d-822c661a7b13 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:34:42.510581 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee6e4c-b29c-4358-99d7-d9f5c749b357 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '627171bf2a2e4a21ba6f800879f8c665', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:34:42.511210 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee6e4c-b29c-4358-99d7-d9f5c749b357 service glance] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:42.511486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee6e4c-b29c-4358-99d7-d9f5c749b357 service glance] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:42.511751 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee6e4c-b29c-4358-99d7-d9f5c749b357 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:42.513354 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-69ee6e4c-b29c-4358-99d7-d9f5c749b357 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:42.585788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:42.626833 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ee6e4c-b29c-4358-99d7-d9f5c749b357 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:34:42.628303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ee6e4c-b29c-4358-99d7-d9f5c749b357 service glance] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:34:42.646508 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 694/1396] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:34:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:34:43.527896 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ef3b8-7e62-4c7e-be5f-2f86f33f4427 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:43.528389 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ef3b8-7e62-4c7e-be5f-2f86f33f4427 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:43.528635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ef3b8-7e62-4c7e-be5f-2f86f33f4427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:43.595058 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ff9ef3b8-7e62-4c7e-be5f-2f86f33f4427 None None] MFA Rules not processed for user `7f3422c6f9a041ceba851b71bd15d0df`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:34:43.596165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ff9ef3b8-7e62-4c7e-be5f-2f86f33f4427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:43.686064 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 703/1397] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 08:34:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:34:43.730509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10ee44-6190-4d72-94e7-533a19cfc055 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:43.730976 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10ee44-6190-4d72-94e7-533a19cfc055 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:43.731102 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10ee44-6190-4d72-94e7-533a19cfc055 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:43.732411 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 695/1398] 10.222.0.33 () {54 vars in 943 bytes} [Mon Aug 8 08:34:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:34:43.738680 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eba323-7657-437c-8d07-1b672fd17a47 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:43.738968 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eba323-7657-437c-8d07-1b672fd17a47 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:43.739240 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eba323-7657-437c-8d07-1b672fd17a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:43.810563 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:43.818542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c9eba323-7657-437c-8d07-1b672fd17a47 None None] MFA Rules not processed for user `e69c0e02ad944f2a85fc5318018e5cb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:34:43.819750 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c9eba323-7657-437c-8d07-1b672fd17a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:43.827859 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359916c0-0223-40f8-bcc9-3fac13f19ec2 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:43.828142 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359916c0-0223-40f8-bcc9-3fac13f19ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:43.828455 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359916c0-0223-40f8-bcc9-3fac13f19ec2 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:43.830457 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 696/1399] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:34:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:34:43.836010 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8ff20-6430-45fe-bb95-f5fe003de8e0 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:43.836294 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8ff20-6430-45fe-bb95-f5fe003de8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:43.836570 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8ff20-6430-45fe-bb95-f5fe003de8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:43.904720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-4ec8ff20-6430-45fe-bb95-f5fe003de8e0 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:34:43.905973 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-4ec8ff20-6430-45fe-bb95-f5fe003de8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:43.936506 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 704/1400] 10.222.0.33 () {58 vars in 1040 bytes} [Mon Aug 8 08:34:43 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:34:43.951706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eba323-7657-437c-8d07-1b672fd17a47 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:34:43.980951 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eba323-7657-437c-8d07-1b672fd17a47 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:34:44.024268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 697/1401] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:34:44.072474 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8ff20-6430-45fe-bb95-f5fe003de8e0 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:34:44.073937 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-4ec8ff20-6430-45fe-bb95-f5fe003de8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:44.102224 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:44.102451 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5771f86-8255-41d4-b6c8-a0e6e4a48806 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e69c0e02ad944f2a85fc5318018e5cb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:34:44.102859 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5771f86-8255-41d4-b6c8-a0e6e4a48806 service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:44.103132 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5771f86-8255-41d4-b6c8-a0e6e4a48806 service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:44.103401 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5771f86-8255-41d4-b6c8-a0e6e4a48806 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:44.105013 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e5771f86-8255-41d4-b6c8-a0e6e4a48806 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:44.107585 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:44.193021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5771f86-8255-41d4-b6c8-a0e6e4a48806 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:34:44.193353 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8ff20-6430-45fe-bb95-f5fe003de8e0 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:34:44.194562 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5771f86-8255-41d4-b6c8-a0e6e4a48806 service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:34:44.204716 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 705/1402] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 08:34:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 08:34:44.312550 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c4003f-acd2-4ec0-b52f-3ed2d9994d13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:34:44.313378 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4003f-acd2-4ec0-b52f-3ed2d9994d13 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:44.313664 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4003f-acd2-4ec0-b52f-3ed2d9994d13 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:44.313930 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4003f-acd2-4ec0-b52f-3ed2d9994d13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:44.315501 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-79c4003f-acd2-4ec0-b52f-3ed2d9994d13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:44.333818 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:44.404420 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c4003f-acd2-4ec0-b52f-3ed2d9994d13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:34:44.406052 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c4003f-acd2-4ec0-b52f-3ed2d9994d13 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:34:44.427091 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 698/1403] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:34:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:34:46.638147 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa98d84-05f5-4206-81f4-4bb4b923660e None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:46.638683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa98d84-05f5-4206-81f4-4bb4b923660e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:46.638683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa98d84-05f5-4206-81f4-4bb4b923660e None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:46.641754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 706/1404] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:34:46.646959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76663662-9b01-49e5-abfd-441ea3cd15a0 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:46.647424 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76663662-9b01-49e5-abfd-441ea3cd15a0 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:46.647508 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76663662-9b01-49e5-abfd-441ea3cd15a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:46.742833 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-76663662-9b01-49e5-abfd-441ea3cd15a0 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:34:46.744291 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-76663662-9b01-49e5-abfd-441ea3cd15a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:46.851661 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:46.858404 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 699/1405] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:34:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:34:46.903535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa98d84-05f5-4206-81f4-4bb4b923660e None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:34:46.905079 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3aa98d84-05f5-4206-81f4-4bb4b923660e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:47.071226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:47.073758 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa98d84-05f5-4206-81f4-4bb4b923660e None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:34:47.203040 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4e33d-d2d3-4d2e-a781-ef2d387b8d2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:34:47.203788 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4e33d-d2d3-4d2e-a781-ef2d387b8d2c service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:47.204122 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4e33d-d2d3-4d2e-a781-ef2d387b8d2c service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:47.204523 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4e33d-d2d3-4d2e-a781-ef2d387b8d2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:47.206292 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-71e4e33d-d2d3-4d2e-a781-ef2d387b8d2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:47.307477 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4e33d-d2d3-4d2e-a781-ef2d387b8d2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:34:47.309227 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4e33d-d2d3-4d2e-a781-ef2d387b8d2c service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:34:47.333416 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 707/1406] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:34:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:34:49.585041 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051158ea-21b1-4585-bc3e-ac30f0f9372c None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:49.585653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051158ea-21b1-4585-bc3e-ac30f0f9372c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:49.585653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051158ea-21b1-4585-bc3e-ac30f0f9372c None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:49.587080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 700/1407] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:34:49.592384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4b9e8-30c2-4ffe-a4b7-ff50ba68d99f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:49.592819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4b9e8-30c2-4ffe-a4b7-ff50ba68d99f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:49.592920 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4b9e8-30c2-4ffe-a4b7-ff50ba68d99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:49.598139 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:49.664908 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ddb4b9e8-30c2-4ffe-a4b7-ff50ba68d99f None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:34:49.666246 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ddb4b9e8-30c2-4ffe-a4b7-ff50ba68d99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:49.777842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 708/1408] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:34:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:34:49.822587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051158ea-21b1-4585-bc3e-ac30f0f9372c None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:34:49.824016 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-051158ea-21b1-4585-bc3e-ac30f0f9372c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:49.990541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051158ea-21b1-4585-bc3e-ac30f0f9372c None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:34:50.064618 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:34:50.115334 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22462f4-f77b-44de-b684-01a713533f96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:34:50.115952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22462f4-f77b-44de-b684-01a713533f96 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:34:50.116232 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22462f4-f77b-44de-b684-01a713533f96 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:34:50.116504 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22462f4-f77b-44de-b684-01a713533f96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:34:50.118194 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d22462f4-f77b-44de-b684-01a713533f96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:34:50.219780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22462f4-f77b-44de-b684-01a713533f96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:34:50.221436 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22462f4-f77b-44de-b684-01a713533f96 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:34:50.242837 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 701/1409] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:34:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:34:59.804148 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=64 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:00.237859 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=64 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:07.574129 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a38b66-3db2-4bb3-a3bf-152fb27c60ed None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:07.574653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a38b66-3db2-4bb3-a3bf-152fb27c60ed None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:07.574653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a38b66-3db2-4bb3-a3bf-152fb27c60ed None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:07.575967 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 709/1410] 10.222.0.33 () {54 vars in 918 bytes} [Mon Aug 8 08:35:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:35:07.581656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23de75-c68d-43b9-b1d5-6637f88f81ad None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:07.586362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23de75-c68d-43b9-b1d5-6637f88f81ad None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:07.586362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23de75-c68d-43b9-b1d5-6637f88f81ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:07.659131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-3d23de75-c68d-43b9-b1d5-6637f88f81ad None None] MFA Rules not processed for user `98c23a782b9b4144bace1248c3b2e754`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:35:07.660333 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-3d23de75-c68d-43b9-b1d5-6637f88f81ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:07.763335 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 702/1411] 10.222.0.33 () {58 vars in 1015 bytes} [Mon Aug 8 08:35:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:35:07.778533 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec975b2a-cef0-42a1-a060-895c44406508 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:07.778817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec975b2a-cef0-42a1-a060-895c44406508 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:07.779090 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec975b2a-cef0-42a1-a060-895c44406508 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:07.780421 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 710/1412] 10.222.0.33 () {54 vars in 918 bytes} [Mon Aug 8 08:35:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:35:07.785702 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486e970-2c4f-46b0-a47d-2c38188a005b None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:07.785992 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486e970-2c4f-46b0-a47d-2c38188a005b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:07.786268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486e970-2c4f-46b0-a47d-2c38188a005b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:07.858542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:07.861837 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-0486e970-2c4f-46b0-a47d-2c38188a005b None None] MFA Rules not processed for user `a3317f751ec74bde95ec262394001292`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:35:07.863032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0486e970-2c4f-46b0-a47d-2c38188a005b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:07.976267 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 703/1413] 10.222.0.33 () {58 vars in 1015 bytes} [Mon Aug 8 08:35:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:35:10.445770 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e6510-ec19-4fba-b5ba-f2e1112a2fb0 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:10.446296 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e6510-ec19-4fba-b5ba-f2e1112a2fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:10.446296 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e6510-ec19-4fba-b5ba-f2e1112a2fb0 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:10.447684 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 08:35:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:35:10.452969 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd6625-a67e-4e7c-82dd-118b7090086a None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:10.453432 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd6625-a67e-4e7c-82dd-118b7090086a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:10.453556 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd6625-a67e-4e7c-82dd-118b7090086a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:10.520434 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-b2dd6625-a67e-4e7c-82dd-118b7090086a None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:35:10.521672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b2dd6625-a67e-4e7c-82dd-118b7090086a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:10.577320 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:10.620945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 08:35:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:35:10.665278 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e6510-ec19-4fba-b5ba-f2e1112a2fb0 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:35:10.671017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-906e6510-ec19-4fba-b5ba-f2e1112a2fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:10.814836 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e6510-ec19-4fba-b5ba-f2e1112a2fb0 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:35:10.885969 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:10.926588 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a119562-c745-4168-9886-6727dd583cf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:35:10.927127 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a119562-c745-4168-9886-6727dd583cf6 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:10.927405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a119562-c745-4168-9886-6727dd583cf6 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:10.927676 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a119562-c745-4168-9886-6727dd583cf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:10.929313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0a119562-c745-4168-9886-6727dd583cf6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:11.016233 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a119562-c745-4168-9886-6727dd583cf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:35:11.018032 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a119562-c745-4168-9886-6727dd583cf6 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:35:11.036185 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 712/1416] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:35:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:35:11.933584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73692e9-5224-41fc-9714-682bf6273580 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:11.934172 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73692e9-5224-41fc-9714-682bf6273580 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:11.934172 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73692e9-5224-41fc-9714-682bf6273580 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:11.935477 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 08:35:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:35:11.940458 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e2e5e-2d68-4120-b783-74fe109558ad None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:11.940863 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e2e5e-2d68-4120-b783-74fe109558ad None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:11.941002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e2e5e-2d68-4120-b783-74fe109558ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:12.015488 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e22e2e5e-2d68-4120-b783-74fe109558ad None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:35:12.017049 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e22e2e5e-2d68-4120-b783-74fe109558ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:12.043487 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:12.112285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 08:35:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:35:12.261551 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73692e9-5224-41fc-9714-682bf6273580 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:35:12.262993 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f73692e9-5224-41fc-9714-682bf6273580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:12.406349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73692e9-5224-41fc-9714-682bf6273580 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:35:12.448630 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:12.520430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47105c-7972-4b50-a749-df92de7b08d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:35:12.520951 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47105c-7972-4b50-a749-df92de7b08d9 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:12.521262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47105c-7972-4b50-a749-df92de7b08d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:12.521532 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47105c-7972-4b50-a749-df92de7b08d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:12.523116 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-df47105c-7972-4b50-a749-df92de7b08d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:12.610876 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47105c-7972-4b50-a749-df92de7b08d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:35:12.612458 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47105c-7972-4b50-a749-df92de7b08d9 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:35:12.630299 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 706/1419] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:35:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:35:13.506468 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e1e4a-aa33-4ebc-a39d-6e240e963cde None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:13.506981 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e1e4a-aa33-4ebc-a39d-6e240e963cde None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:13.506981 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e1e4a-aa33-4ebc-a39d-6e240e963cde None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:13.508302 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 08:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:35:13.513061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12added-751d-4109-ad3f-556c5a4342cf None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:13.513527 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12added-751d-4109-ad3f-556c5a4342cf None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:13.513633 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12added-751d-4109-ad3f-556c5a4342cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:13.569189 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:13.590742 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-b12added-751d-4109-ad3f-556c5a4342cf None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:35:13.592054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b12added-751d-4109-ad3f-556c5a4342cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:13.687653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 08:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:35:13.841736 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e1e4a-aa33-4ebc-a39d-6e240e963cde None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:35:13.843313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ec2e1e4a-aa33-4ebc-a39d-6e240e963cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:13.993249 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e1e4a-aa33-4ebc-a39d-6e240e963cde None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:35:14.005625 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:14.104421 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa546a-b248-4754-a303-ae2817383dee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:35:14.104938 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa546a-b248-4754-a303-ae2817383dee service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:14.105241 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa546a-b248-4754-a303-ae2817383dee service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:14.105511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa546a-b248-4754-a303-ae2817383dee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:14.107203 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ebaa546a-b248-4754-a303-ae2817383dee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:14.197603 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa546a-b248-4754-a303-ae2817383dee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:35:14.199214 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa546a-b248-4754-a303-ae2817383dee service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:35:14.217762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 715/1422] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:35:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:35:15.082122 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d965a-c55d-47fa-9f85-a34abbf7e875 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:15.082687 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d965a-c55d-47fa-9f85-a34abbf7e875 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:15.082687 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d965a-c55d-47fa-9f85-a34abbf7e875 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:15.084102 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 08:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:35:15.089285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877cdcf-f8de-4b74-bab2-9e07e85496cc None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:15.089730 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877cdcf-f8de-4b74-bab2-9e07e85496cc None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:15.089841 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877cdcf-f8de-4b74-bab2-9e07e85496cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:15.109645 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:15.163929 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0877cdcf-f8de-4b74-bab2-9e07e85496cc None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:35:15.165116 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0877cdcf-f8de-4b74-bab2-9e07e85496cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:15.258671 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 08:35:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:35:15.411216 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d965a-c55d-47fa-9f85-a34abbf7e875 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:35:15.412825 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-df6d965a-c55d-47fa-9f85-a34abbf7e875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:15.520181 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:15.557282 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d965a-c55d-47fa-9f85-a34abbf7e875 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:35:15.669802 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649b583-8505-4459-873a-2c6b5d5fcbc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:35:15.670330 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649b583-8505-4459-873a-2c6b5d5fcbc5 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:15.670612 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649b583-8505-4459-873a-2c6b5d5fcbc5 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:15.670881 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649b583-8505-4459-873a-2c6b5d5fcbc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:15.673234 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b649b583-8505-4459-873a-2c6b5d5fcbc5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:15.747326 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:15.760496 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649b583-8505-4459-873a-2c6b5d5fcbc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:35:15.762241 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649b583-8505-4459-873a-2c6b5d5fcbc5 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:35:15.779947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 709/1425] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:35:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:35:16.654726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef6961-46a1-4e52-8f08-bf52adf49109 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:16.655308 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef6961-46a1-4e52-8f08-bf52adf49109 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:16.655308 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef6961-46a1-4e52-8f08-bf52adf49109 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:16.657221 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 08:35:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:35:16.662139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44365352-73b8-4c8e-8828-9d9762ff6d2f None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:16.662584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44365352-73b8-4c8e-8828-9d9762ff6d2f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:16.662679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44365352-73b8-4c8e-8828-9d9762ff6d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:16.736902 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-44365352-73b8-4c8e-8828-9d9762ff6d2f None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:35:16.738136 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-44365352-73b8-4c8e-8828-9d9762ff6d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:16.832500 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 08:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:35:16.982655 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef6961-46a1-4e52-8f08-bf52adf49109 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:35:16.984172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-74ef6961-46a1-4e52-8f08-bf52adf49109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:17.068140 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:17.129415 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef6961-46a1-4e52-8f08-bf52adf49109 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:35:17.240760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187c0769-0048-4648-ae11-3696e2b8801e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:35:17.241393 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c0769-0048-4648-ae11-3696e2b8801e service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:17.241674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c0769-0048-4648-ae11-3696e2b8801e service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:17.241944 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c0769-0048-4648-ae11-3696e2b8801e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:17.243540 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-187c0769-0048-4648-ae11-3696e2b8801e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:17.291339 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:17.332007 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187c0769-0048-4648-ae11-3696e2b8801e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:35:17.333579 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187c0769-0048-4648-ae11-3696e2b8801e service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:35:17.351240 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 718/1428] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:35:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:35:18.208262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959f490-d238-4d99-869d-eaef48f67290 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:18.208810 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959f490-d238-4d99-869d-eaef48f67290 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:18.208810 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959f490-d238-4d99-869d-eaef48f67290 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:18.210476 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 08:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:35:18.215303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655013c-9d4e-46ed-8a26-173231f552f4 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:18.215736 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655013c-9d4e-46ed-8a26-173231f552f4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:18.215839 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655013c-9d4e-46ed-8a26-173231f552f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:18.290540 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0655013c-9d4e-46ed-8a26-173231f552f4 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:35:18.291728 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0655013c-9d4e-46ed-8a26-173231f552f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:18.387357 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 08:35:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:35:18.538425 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959f490-d238-4d99-869d-eaef48f67290 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:35:18.539911 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6959f490-d238-4d99-869d-eaef48f67290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:18.588818 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:18.687819 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959f490-d238-4d99-869d-eaef48f67290 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:35:18.800606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c454d-fef8-440a-bced-13b6c083a89a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:35:18.801123 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c454d-fef8-440a-bced-13b6c083a89a service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:18.801430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c454d-fef8-440a-bced-13b6c083a89a service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:18.801698 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c454d-fef8-440a-bced-13b6c083a89a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:18.803305 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1b6c454d-fef8-440a-bced-13b6c083a89a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:18.810119 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:18.890669 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c454d-fef8-440a-bced-13b6c083a89a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:35:18.892907 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c454d-fef8-440a-bced-13b6c083a89a service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:35:18.911244 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 712/1431] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:35:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:35:19.803196 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8dfc58-feb7-418c-bed9-8cb36d8c3c43 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:19.803805 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8dfc58-feb7-418c-bed9-8cb36d8c3c43 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:19.803805 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8dfc58-feb7-418c-bed9-8cb36d8c3c43 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:19.805164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 08:35:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:35:19.810302 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908c570-d8a5-4b83-8895-4efbd467d06b None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:19.810735 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908c570-d8a5-4b83-8895-4efbd467d06b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:19.810843 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908c570-d8a5-4b83-8895-4efbd467d06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:19.887508 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-2908c570-d8a5-4b83-8895-4efbd467d06b None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:35:19.888718 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2908c570-d8a5-4b83-8895-4efbd467d06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:19.954972 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:19.985239 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 08:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:35:20.139743 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8dfc58-feb7-418c-bed9-8cb36d8c3c43 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:35:20.141345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0c8dfc58-feb7-418c-bed9-8cb36d8c3c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:20.160047 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:20.291619 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8dfc58-feb7-418c-bed9-8cb36d8c3c43 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:35:20.379280 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:20.404933 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da06e8-253b-4343-a184-8f6b69309a7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:35:20.405500 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da06e8-253b-4343-a184-8f6b69309a7d service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:20.405781 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da06e8-253b-4343-a184-8f6b69309a7d service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:20.406052 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da06e8-253b-4343-a184-8f6b69309a7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:20.408732 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a9da06e8-253b-4343-a184-8f6b69309a7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:20.497683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9da06e8-253b-4343-a184-8f6b69309a7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:35:20.499279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9da06e8-253b-4343-a184-8f6b69309a7d service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:35:20.517336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 721/1434] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:35:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:35:21.415400 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb5a8f-4c98-4310-833b-72b50999db43 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:21.415910 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb5a8f-4c98-4310-833b-72b50999db43 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:21.415910 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb5a8f-4c98-4310-833b-72b50999db43 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:21.417242 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 08:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:35:21.422168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83ad85-2a17-4e6e-8eca-2bf016c47c3a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:21.422625 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83ad85-2a17-4e6e-8eca-2bf016c47c3a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:21.422831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83ad85-2a17-4e6e-8eca-2bf016c47c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:21.497623 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4d83ad85-2a17-4e6e-8eca-2bf016c47c3a None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:35:21.498824 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4d83ad85-2a17-4e6e-8eca-2bf016c47c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:21.540720 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:21.595648 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 08:35:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:35:21.745246 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb5a8f-4c98-4310-833b-72b50999db43 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:35:21.746674 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-28bb5a8f-4c98-4310-833b-72b50999db43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:21.888455 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb5a8f-4c98-4310-833b-72b50999db43 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:35:21.942667 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:22.000652 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481900e3-2bc7-4edc-ae5d-9326cbe26325 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:35:22.001283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481900e3-2bc7-4edc-ae5d-9326cbe26325 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:22.001568 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481900e3-2bc7-4edc-ae5d-9326cbe26325 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:22.001835 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481900e3-2bc7-4edc-ae5d-9326cbe26325 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:22.003436 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-481900e3-2bc7-4edc-ae5d-9326cbe26325 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:22.093101 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481900e3-2bc7-4edc-ae5d-9326cbe26325 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:35:22.095040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481900e3-2bc7-4edc-ae5d-9326cbe26325 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:35:22.113599 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 715/1437] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:35:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:35:22.979450 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5f974-abe7-483f-849b-74aba8e03933 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:22.980029 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5f974-abe7-483f-849b-74aba8e03933 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:22.980029 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5f974-abe7-483f-849b-74aba8e03933 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:22.981355 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 08:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:35:22.987113 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df807503-542f-48d4-a313-cb83bc55e464 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:22.987546 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df807503-542f-48d4-a313-cb83bc55e464 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:22.987660 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df807503-542f-48d4-a313-cb83bc55e464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:23.063539 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-df807503-542f-48d4-a313-cb83bc55e464 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:35:23.064746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-df807503-542f-48d4-a313-cb83bc55e464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:23.066735 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:23.160290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 08:35:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:35:23.311421 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5f974-abe7-483f-849b-74aba8e03933 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:35:23.312940 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-43c5f974-abe7-483f-849b-74aba8e03933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:23.460008 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5f974-abe7-483f-849b-74aba8e03933 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:35:23.486931 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:23.574612 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa663df6-5a48-4e9f-a26f-f68f6873dff1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:35:23.575171 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa663df6-5a48-4e9f-a26f-f68f6873dff1 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:23.575450 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa663df6-5a48-4e9f-a26f-f68f6873dff1 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:23.575718 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa663df6-5a48-4e9f-a26f-f68f6873dff1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:23.577375 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-aa663df6-5a48-4e9f-a26f-f68f6873dff1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:23.668504 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa663df6-5a48-4e9f-a26f-f68f6873dff1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:35:23.670132 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa663df6-5a48-4e9f-a26f-f68f6873dff1 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:35:23.688402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 724/1440] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:35:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:35:25.588936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fb78c-4616-4fd5-83eb-bccc24bc8808 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:25.589665 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fb78c-4616-4fd5-83eb-bccc24bc8808 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:25.589665 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fb78c-4616-4fd5-83eb-bccc24bc8808 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:25.590957 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 08:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:35:25.596774 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f4488-c134-44a5-bb7b-79c7de09b2a2 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:25.597199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f4488-c134-44a5-bb7b-79c7de09b2a2 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:25.599647 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f4488-c134-44a5-bb7b-79c7de09b2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:25.637331 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:25.663994 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-de0f4488-c134-44a5-bb7b-79c7de09b2a2 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:35:25.665269 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-de0f4488-c134-44a5-bb7b-79c7de09b2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:25.767565 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 08:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:35:25.806437 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2fb78c-4616-4fd5-83eb-bccc24bc8808 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:35:25.807944 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ce2fb78c-4616-4fd5-83eb-bccc24bc8808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:25.909391 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2fb78c-4616-4fd5-83eb-bccc24bc8808 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:35:25.940614 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:25.993506 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a009b-96f9-4632-8464-08c071f9b78b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:35:25.994120 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a009b-96f9-4632-8464-08c071f9b78b service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:25.994395 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a009b-96f9-4632-8464-08c071f9b78b service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:25.994665 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a009b-96f9-4632-8464-08c071f9b78b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:25.996253 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-846a009b-96f9-4632-8464-08c071f9b78b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:26.103232 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846a009b-96f9-4632-8464-08c071f9b78b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:35:26.104579 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846a009b-96f9-4632-8464-08c071f9b78b service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:35:26.126364 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 718/1443] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:35:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:35:28.270363 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508a043-2243-431c-9721-e3dd22a7a40f None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:28.270911 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508a043-2243-431c-9721-e3dd22a7a40f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:28.270911 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508a043-2243-431c-9721-e3dd22a7a40f None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:28.272225 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 08:35:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:35:28.277419 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6af486-8c11-4ae5-b276-92362c248e49 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:28.277864 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6af486-8c11-4ae5-b276-92362c248e49 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:28.277984 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6af486-8c11-4ae5-b276-92362c248e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:28.342444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-bd6af486-8c11-4ae5-b276-92362c248e49 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:35:28.343655 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-bd6af486-8c11-4ae5-b276-92362c248e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:28.356194 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:28.442332 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 08:35:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:35:28.480536 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508a043-2243-431c-9721-e3dd22a7a40f None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:35:28.482069 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f508a043-2243-431c-9721-e3dd22a7a40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:28.584471 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508a043-2243-431c-9721-e3dd22a7a40f None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:35:28.596590 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:28.657861 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9f7ed-53c3-4e6a-84e6-a9d042208c93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:35:28.658469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9f7ed-53c3-4e6a-84e6-a9d042208c93 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:28.658754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9f7ed-53c3-4e6a-84e6-a9d042208c93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:28.659026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9f7ed-53c3-4e6a-84e6-a9d042208c93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:28.660733 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-42b9f7ed-53c3-4e6a-84e6-a9d042208c93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:28.771491 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b9f7ed-53c3-4e6a-84e6-a9d042208c93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:35:28.772918 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b9f7ed-53c3-4e6a-84e6-a9d042208c93 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:35:28.790862 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 727/1446] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:35:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:35:31.166804 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c8c1a-5302-4c40-80dd-bec8f6e0625b None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:31.167394 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c8c1a-5302-4c40-80dd-bec8f6e0625b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:31.167394 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c8c1a-5302-4c40-80dd-bec8f6e0625b None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:31.168707 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 08:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:35:31.174206 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c10e6-0dde-4b76-85eb-d8f3c2953b77 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:31.174689 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c10e6-0dde-4b76-85eb-d8f3c2953b77 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:31.174800 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c10e6-0dde-4b76-85eb-d8f3c2953b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:31.223056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:31.240412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-527c10e6-0dde-4b76-85eb-d8f3c2953b77 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:35:31.244051 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-527c10e6-0dde-4b76-85eb-d8f3c2953b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:31.345204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 08:35:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:35:31.384395 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c8c1a-5302-4c40-80dd-bec8f6e0625b None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:35:31.385931 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-522c8c1a-5302-4c40-80dd-bec8f6e0625b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:31.488324 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c8c1a-5302-4c40-80dd-bec8f6e0625b None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:35:31.528445 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:31.561899 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e17db-4716-4de1-80a3-e85e88882dea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:35:31.563111 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e17db-4716-4de1-80a3-e85e88882dea service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:31.563384 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e17db-4716-4de1-80a3-e85e88882dea service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:31.563651 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e17db-4716-4de1-80a3-e85e88882dea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:31.565342 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-7f1e17db-4716-4de1-80a3-e85e88882dea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:31.670658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e17db-4716-4de1-80a3-e85e88882dea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:35:31.672751 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e17db-4716-4de1-80a3-e85e88882dea service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:35:31.690513 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 721/1449] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:35:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:35:33.805552 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0752c9-6a91-405c-b2c0-9dffd198c07c None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:33.806171 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0752c9-6a91-405c-b2c0-9dffd198c07c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:33.806171 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0752c9-6a91-405c-b2c0-9dffd198c07c None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:33.807481 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 08:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:35:33.812657 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3df96-b8fc-4249-894f-8859293f0221 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:33.813274 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3df96-b8fc-4249-894f-8859293f0221 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:33.813274 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3df96-b8fc-4249-894f-8859293f0221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:33.878503 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-73d3df96-b8fc-4249-894f-8859293f0221 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:35:33.879753 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-73d3df96-b8fc-4249-894f-8859293f0221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:33.900412 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:33.977623 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 08:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:35:34.016721 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0752c9-6a91-405c-b2c0-9dffd198c07c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:35:34.018235 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-eb0752c9-6a91-405c-b2c0-9dffd198c07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:34.120852 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0752c9-6a91-405c-b2c0-9dffd198c07c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:35:34.141578 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:34.195417 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b1e543-d9e1-4bab-b2a1-4c63af691966 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:35:34.196021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1e543-d9e1-4bab-b2a1-4c63af691966 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:34.196298 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1e543-d9e1-4bab-b2a1-4c63af691966 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:34.196569 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1e543-d9e1-4bab-b2a1-4c63af691966 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:34.198198 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-16b1e543-d9e1-4bab-b2a1-4c63af691966 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:34.305658 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b1e543-d9e1-4bab-b2a1-4c63af691966 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:35:34.307036 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b1e543-d9e1-4bab-b2a1-4c63af691966 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:35:34.324839 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 730/1452] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:35:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:35:36.385807 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a87c1-8164-4427-bb07-5d88783f1874 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:36.386365 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a87c1-8164-4427-bb07-5d88783f1874 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:36.386365 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a87c1-8164-4427-bb07-5d88783f1874 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:36.387707 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 08:35:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:35:36.393767 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b900c-a004-4f13-93c2-856f6ca543ae None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:36.394207 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b900c-a004-4f13-93c2-856f6ca543ae None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:36.394313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b900c-a004-4f13-93c2-856f6ca543ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:36.457821 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:36.460666 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-fa7b900c-a004-4f13-93c2-856f6ca543ae None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:35:36.461916 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fa7b900c-a004-4f13-93c2-856f6ca543ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:36.556222 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 08:35:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:35:36.594747 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a87c1-8164-4427-bb07-5d88783f1874 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:35:36.596275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-211a87c1-8164-4427-bb07-5d88783f1874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:36.697903 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a87c1-8164-4427-bb07-5d88783f1874 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:35:36.745649 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:36.770471 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c506f85f-df0c-4deb-847c-43b453f60561 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:35:36.771109 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506f85f-df0c-4deb-847c-43b453f60561 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:36.771386 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506f85f-df0c-4deb-847c-43b453f60561 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:36.771652 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506f85f-df0c-4deb-847c-43b453f60561 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:36.773249 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c506f85f-df0c-4deb-847c-43b453f60561 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:36.878132 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c506f85f-df0c-4deb-847c-43b453f60561 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:35:36.879539 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c506f85f-df0c-4deb-847c-43b453f60561 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:35:36.899139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 724/1455] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:35:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:35:38.957671 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b89b03-4a0e-4580-bdad-8c214bcb30a1 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:38.958243 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b89b03-4a0e-4580-bdad-8c214bcb30a1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:38.958243 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b89b03-4a0e-4580-bdad-8c214bcb30a1 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:38.959581 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 08:35:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:35:38.964915 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd646c-c1f7-4ea7-82eb-ff7c4df44266 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:38.965350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd646c-c1f7-4ea7-82eb-ff7c4df44266 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:38.965522 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd646c-c1f7-4ea7-82eb-ff7c4df44266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:39.151320 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-3efd646c-c1f7-4ea7-82eb-ff7c4df44266 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:35:39.152529 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-3efd646c-c1f7-4ea7-82eb-ff7c4df44266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:39.181249 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:39.250029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 08:35:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:35:39.287983 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b89b03-4a0e-4580-bdad-8c214bcb30a1 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:35:39.289549 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f5b89b03-4a0e-4580-bdad-8c214bcb30a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:39.393991 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b89b03-4a0e-4580-bdad-8c214bcb30a1 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:35:39.423049 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:39.467492 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f196dc-956a-4018-9c09-2a6b56763582 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:35:39.468090 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f196dc-956a-4018-9c09-2a6b56763582 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:39.468862 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f196dc-956a-4018-9c09-2a6b56763582 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:39.469122 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f196dc-956a-4018-9c09-2a6b56763582 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:39.470818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-82f196dc-956a-4018-9c09-2a6b56763582 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:39.576883 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f196dc-956a-4018-9c09-2a6b56763582 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:35:39.578440 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f196dc-956a-4018-9c09-2a6b56763582 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:35:39.597814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 733/1458] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:35:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:35:41.694440 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69aadaa-5bc0-45a4-920e-ebef378f6dd0 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:41.695002 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69aadaa-5bc0-45a4-920e-ebef378f6dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:41.695002 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69aadaa-5bc0-45a4-920e-ebef378f6dd0 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:41.696338 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 08:35:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:35:41.701466 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a43c9b-988e-4e8a-b502-88a1f7d18f5a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:41.701867 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a43c9b-988e-4e8a-b502-88a1f7d18f5a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:41.702016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a43c9b-988e-4e8a-b502-88a1f7d18f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:41.770287 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a4a43c9b-988e-4e8a-b502-88a1f7d18f5a None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:35:41.771480 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a4a43c9b-988e-4e8a-b502-88a1f7d18f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:41.779536 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:41.986878 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 08:35:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:35:42.025080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69aadaa-5bc0-45a4-920e-ebef378f6dd0 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:35:42.026614 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a69aadaa-5bc0-45a4-920e-ebef378f6dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:42.129702 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69aadaa-5bc0-45a4-920e-ebef378f6dd0 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:35:42.183574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:42.201772 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe31ccb-8372-434f-b15a-f575227c5fb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:35:42.202381 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe31ccb-8372-434f-b15a-f575227c5fb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:42.202770 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe31ccb-8372-434f-b15a-f575227c5fb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:42.203045 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe31ccb-8372-434f-b15a-f575227c5fb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:42.204731 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-dbe31ccb-8372-434f-b15a-f575227c5fb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:42.311174 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe31ccb-8372-434f-b15a-f575227c5fb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:35:42.312758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe31ccb-8372-434f-b15a-f575227c5fb1 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:35:42.330766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 727/1461] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:35:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:35:44.359576 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87446754-43de-4f2c-adee-6b28f7a49e8c None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:44.360153 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87446754-43de-4f2c-adee-6b28f7a49e8c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:44.360153 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87446754-43de-4f2c-adee-6b28f7a49e8c None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:44.361509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 08:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:35:44.366807 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b726bb-b6ee-4cf0-8f4c-fbbe5a3f4c0e None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:44.367204 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b726bb-b6ee-4cf0-8f4c-fbbe5a3f4c0e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:44.367353 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b726bb-b6ee-4cf0-8f4c-fbbe5a3f4c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:44.434444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-57b726bb-b6ee-4cf0-8f4c-fbbe5a3f4c0e None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:35:44.435627 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-57b726bb-b6ee-4cf0-8f4c-fbbe5a3f4c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:44.472850 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:44.529263 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 08:35:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:35:44.567010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87446754-43de-4f2c-adee-6b28f7a49e8c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:35:44.568544 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-87446754-43de-4f2c-adee-6b28f7a49e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:44.669398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87446754-43de-4f2c-adee-6b28f7a49e8c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:35:44.706327 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:44.741329 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893afbdb-1eaf-4399-b22c-d78b41ecb3e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:35:44.741927 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893afbdb-1eaf-4399-b22c-d78b41ecb3e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:44.742198 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893afbdb-1eaf-4399-b22c-d78b41ecb3e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:44.742474 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893afbdb-1eaf-4399-b22c-d78b41ecb3e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:44.744013 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-893afbdb-1eaf-4399-b22c-d78b41ecb3e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:44.849562 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893afbdb-1eaf-4399-b22c-d78b41ecb3e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:35:44.850990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893afbdb-1eaf-4399-b22c-d78b41ecb3e7 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:35:44.868313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 736/1464] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:35:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:35:46.865169 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbfedf-4151-4691-9b24-84d55ee9427d None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:46.865682 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbfedf-4151-4691-9b24-84d55ee9427d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:46.865682 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbfedf-4151-4691-9b24-84d55ee9427d None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:46.867061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 08:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:35:46.872215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0271c-e2ae-45b5-a538-e87f312bf8fc None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:46.872614 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0271c-e2ae-45b5-a538-e87f312bf8fc None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:46.872759 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0271c-e2ae-45b5-a538-e87f312bf8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:46.938346 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-1fb0271c-e2ae-45b5-a538-e87f312bf8fc None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:35:46.939552 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1fb0271c-e2ae-45b5-a538-e87f312bf8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:46.957430 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:47.038310 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 08:35:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:35:47.074713 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dbfedf-4151-4691-9b24-84d55ee9427d None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:35:47.076259 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e5dbfedf-4151-4691-9b24-84d55ee9427d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:47.178686 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dbfedf-4151-4691-9b24-84d55ee9427d None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:35:47.243997 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:47.250817 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada635f-4b98-4643-888a-654f32f97e5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:35:47.251417 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada635f-4b98-4643-888a-654f32f97e5f service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:47.251691 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada635f-4b98-4643-888a-654f32f97e5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:47.251955 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada635f-4b98-4643-888a-654f32f97e5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:47.253555 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-eada635f-4b98-4643-888a-654f32f97e5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:47.360983 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada635f-4b98-4643-888a-654f32f97e5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:35:47.362435 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada635f-4b98-4643-888a-654f32f97e5f service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:35:47.380848 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 730/1467] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:35:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:35:49.385985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0952ff-d947-416e-a424-0babdcb1627c None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:49.386517 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0952ff-d947-416e-a424-0babdcb1627c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:49.386517 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0952ff-d947-416e-a424-0babdcb1627c None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:49.387854 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 08:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:35:49.393039 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afafb3-4f78-4cbd-bd18-dcb181de0208 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:49.393484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afafb3-4f78-4cbd-bd18-dcb181de0208 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:49.393624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afafb3-4f78-4cbd-bd18-dcb181de0208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:49.459970 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-d4afafb3-4f78-4cbd-bd18-dcb181de0208 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:35:49.461480 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d4afafb3-4f78-4cbd-bd18-dcb181de0208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:49.509563 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:49.558192 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 08:35:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:35:49.596145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0952ff-d947-416e-a424-0babdcb1627c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:35:49.597703 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4d0952ff-d947-416e-a424-0babdcb1627c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:49.700186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0952ff-d947-416e-a424-0babdcb1627c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:35:49.747210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:49.772454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d797f5-4f7e-45f3-a904-5e9c2bf99d96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:35:49.773051 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d797f5-4f7e-45f3-a904-5e9c2bf99d96 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:35:49.773348 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d797f5-4f7e-45f3-a904-5e9c2bf99d96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:35:49.773612 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d797f5-4f7e-45f3-a904-5e9c2bf99d96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:35:49.775171 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-80d797f5-4f7e-45f3-a904-5e9c2bf99d96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:35:49.879931 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d797f5-4f7e-45f3-a904-5e9c2bf99d96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:35:49.881380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d797f5-4f7e-45f3-a904-5e9c2bf99d96 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:35:49.899511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 739/1470] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:35:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:35:59.556340 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=22 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:35:59.897302 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=66 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:36:09.602924 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc29efa-1573-467b-a0aa-ea4733fa90a7 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:36:09.603484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc29efa-1573-467b-a0aa-ea4733fa90a7 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:36:09.603603 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc29efa-1573-467b-a0aa-ea4733fa90a7 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:36:09.604915 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 732/1471] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:36:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:36:09.610832 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c08e3-1854-4783-a159-aac1b26f0b7f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:36:09.611268 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c08e3-1854-4783-a159-aac1b26f0b7f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:36:09.614436 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c08e3-1854-4783-a159-aac1b26f0b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:36:09.676485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e04c08e3-1854-4783-a159-aac1b26f0b7f None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:36:09.677766 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e04c08e3-1854-4783-a159-aac1b26f0b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:36:09.773722 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 740/1472] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:36:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:36:09.856635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9f066-b443-4e75-8048-5fb906174525 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:36:09.856913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9f066-b443-4e75-8048-5fb906174525 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:36:09.857208 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9f066-b443-4e75-8048-5fb906174525 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:36:09.858642 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 733/1473] 10.222.0.33 () {54 vars in 967 bytes} [Mon Aug 8 08:36:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:36:09.863696 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9d76c-7878-4525-bbb0-e42a5345f37a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:36:09.863983 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9d76c-7878-4525-bbb0-e42a5345f37a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:36:09.864259 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9d76c-7878-4525-bbb0-e42a5345f37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:36:09.939899 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-70e9d76c-7878-4525-bbb0-e42a5345f37a None None] MFA Rules not processed for user `627171bf2a2e4a21ba6f800879f8c665`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:36:09.941083 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-70e9d76c-7878-4525-bbb0-e42a5345f37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:36:09.961646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:36:10.016042 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 741/1474] 10.222.0.33 () {58 vars in 1064 bytes} [Mon Aug 8 08:36:09 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:36:10.023545 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e9f066-b443-4e75-8048-5fb906174525 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:36:10.025112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-19e9f066-b443-4e75-8048-5fb906174525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:36:10.127300 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e9f066-b443-4e75-8048-5fb906174525 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:36:10.198758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87560035-c636-44d0-b237-1f6d70667bae service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '627171bf2a2e4a21ba6f800879f8c665', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:36:10.199355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87560035-c636-44d0-b237-1f6d70667bae service glance] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:36:10.199629 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87560035-c636-44d0-b237-1f6d70667bae service glance] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:36:10.199896 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87560035-c636-44d0-b237-1f6d70667bae service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:36:10.201500 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-87560035-c636-44d0-b237-1f6d70667bae service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:36:10.254777 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:36:10.307203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87560035-c636-44d0-b237-1f6d70667bae service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:36:10.308609 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87560035-c636-44d0-b237-1f6d70667bae service glance] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:36:10.327448 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 734/1475] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:36:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:36:12.411884 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f20b99-3571-4a4a-b9f0-78d27fac70da None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:36:12.412446 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f20b99-3571-4a4a-b9f0-78d27fac70da None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:36:12.412446 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f20b99-3571-4a4a-b9f0-78d27fac70da None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:36:12.413792 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 742/1476] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:36:12.419002 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da3786-b659-4304-9cb2-9f4bf83c3f79 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:36:12.419396 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da3786-b659-4304-9cb2-9f4bf83c3f79 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:36:12.419546 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da3786-b659-4304-9cb2-9f4bf83c3f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:36:12.483844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-b6da3786-b659-4304-9cb2-9f4bf83c3f79 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:36:12.485036 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b6da3786-b659-4304-9cb2-9f4bf83c3f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:36:12.581872 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 735/1477] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:36:12.604952 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f20b99-3571-4a4a-b9f0-78d27fac70da None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:36:12.606496 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-89f20b99-3571-4a4a-b9f0-78d27fac70da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:36:12.730945 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f20b99-3571-4a4a-b9f0-78d27fac70da None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:36:12.779678 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:36:12.823993 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d0895-d71f-4ea6-bc6e-3979c2d391b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:36:12.824593 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d0895-d71f-4ea6-bc6e-3979c2d391b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:36:12.824864 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d0895-d71f-4ea6-bc6e-3979c2d391b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:36:12.825130 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d0895-d71f-4ea6-bc6e-3979c2d391b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:36:12.825814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889d0895-d71f-4ea6-bc6e-3979c2d391b4 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:36:12.828048 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: WARNING py.warnings [None req-889d0895-d71f-4ea6-bc6e-3979c2d391b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 08:36:12.828048 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: warnings.warn(msg) Aug 08 08:36:12.828048 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: Aug 08 08:36:12.829225 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889d0895-d71f-4ea6-bc6e-3979c2d391b4 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:36:12.848055 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 743/1478] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 08:36:12 2022] GET /identity/v3/projects => generated 2553 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:36:14.841299 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498283c-40c2-450c-a5bc-5a9331184719 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:36:14.841832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498283c-40c2-450c-a5bc-5a9331184719 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:36:14.841832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498283c-40c2-450c-a5bc-5a9331184719 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:36:14.843750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 736/1479] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:36:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:36:14.849110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee82fb-5cc1-45c6-a064-c6f14102f555 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:36:14.849554 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee82fb-5cc1-45c6-a064-c6f14102f555 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:36:14.849677 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee82fb-5cc1-45c6-a064-c6f14102f555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:36:14.915311 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a1ee82fb-5cc1-45c6-a064-c6f14102f555 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:36:14.916530 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a1ee82fb-5cc1-45c6-a064-c6f14102f555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:36:15.013197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 744/1480] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:36:15.056440 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498283c-40c2-450c-a5bc-5a9331184719 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:36:15.058029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8498283c-40c2-450c-a5bc-5a9331184719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:36:15.067953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:36:15.203891 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498283c-40c2-450c-a5bc-5a9331184719 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:36:15.281636 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:36:15.316958 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211243f-724f-4f7c-a629-51c3bc427cc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:36:15.317604 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211243f-724f-4f7c-a629-51c3bc427cc9 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:36:15.317884 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211243f-724f-4f7c-a629-51c3bc427cc9 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:36:15.318148 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211243f-724f-4f7c-a629-51c3bc427cc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:36:15.319746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-3211243f-724f-4f7c-a629-51c3bc427cc9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:36:15.407296 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3211243f-724f-4f7c-a629-51c3bc427cc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:36:15.408896 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3211243f-724f-4f7c-a629-51c3bc427cc9 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:36:15.427584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 737/1481] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:36:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:36:17.621537 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd52ba5-e86c-4ebc-9f0b-bff8884f8713 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:36:17.622055 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd52ba5-e86c-4ebc-9f0b-bff8884f8713 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:36:17.622055 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd52ba5-e86c-4ebc-9f0b-bff8884f8713 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:36:17.623630 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 745/1482] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:36:17.628497 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc213d66-7527-419f-9508-ed6bafbae208 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:36:17.628936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc213d66-7527-419f-9508-ed6bafbae208 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:36:17.629034 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc213d66-7527-419f-9508-ed6bafbae208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:36:17.694625 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-cc213d66-7527-419f-9508-ed6bafbae208 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:36:17.695831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-cc213d66-7527-419f-9508-ed6bafbae208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:36:17.741239 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:36:17.791384 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 738/1483] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:36:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:36:17.828758 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd52ba5-e86c-4ebc-9f0b-bff8884f8713 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:36:17.830308 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ddd52ba5-e86c-4ebc-9f0b-bff8884f8713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:36:17.853404 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:36:17.932650 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd52ba5-e86c-4ebc-9f0b-bff8884f8713 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:36:18.004643 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122fc526-6a6f-4401-9924-26f2a9df7838 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:36:18.005281 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fc526-6a6f-4401-9924-26f2a9df7838 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:36:18.005562 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fc526-6a6f-4401-9924-26f2a9df7838 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:36:18.005837 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fc526-6a6f-4401-9924-26f2a9df7838 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:36:18.007418 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-122fc526-6a6f-4401-9924-26f2a9df7838 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:36:18.095292 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:36:18.130976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122fc526-6a6f-4401-9924-26f2a9df7838 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:36:18.132415 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122fc526-6a6f-4401-9924-26f2a9df7838 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:36:18.150443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 746/1484] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:36:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:36:20.195977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12b271-1769-4bd3-9163-5c546446bf64 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:36:20.196432 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12b271-1769-4bd3-9163-5c546446bf64 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:36:20.196539 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12b271-1769-4bd3-9163-5c546446bf64 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:36:20.197927 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 739/1485] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:36:20.203221 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab2808-00a1-41e2-928a-9cfcb8552bc8 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:36:20.203633 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab2808-00a1-41e2-928a-9cfcb8552bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:36:20.203748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab2808-00a1-41e2-928a-9cfcb8552bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:36:20.268589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-69ab2808-00a1-41e2-928a-9cfcb8552bc8 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:36:20.269815 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-69ab2808-00a1-41e2-928a-9cfcb8552bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:36:20.365505 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 747/1486] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:36:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:36:20.403816 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12b271-1769-4bd3-9163-5c546446bf64 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:36:20.406204 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-bb12b271-1769-4bd3-9163-5c546446bf64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:36:20.508811 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12b271-1769-4bd3-9163-5c546446bf64 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:36:20.579433 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:36:20.581191 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fae34a-ad13-46c5-9654-4f6400c00949 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:36:20.581794 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fae34a-ad13-46c5-9654-4f6400c00949 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:36:20.582069 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fae34a-ad13-46c5-9654-4f6400c00949 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:36:20.582338 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fae34a-ad13-46c5-9654-4f6400c00949 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:36:20.583918 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-13fae34a-ad13-46c5-9654-4f6400c00949 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:36:20.692274 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fae34a-ad13-46c5-9654-4f6400c00949 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:36:20.693816 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fae34a-ad13-46c5-9654-4f6400c00949 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:36:20.712652 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 740/1487] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:36:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:36:22.859346 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5124c8-e771-4383-828d-6e7624f5f1d6 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:36:22.859872 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5124c8-e771-4383-828d-6e7624f5f1d6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:36:22.859872 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5124c8-e771-4383-828d-6e7624f5f1d6 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:36:22.861166 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 748/1488] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:36:22.866329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e360f3f-25ac-4580-83b3-5426f5a456c7 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:36:22.866708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e360f3f-25ac-4580-83b3-5426f5a456c7 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:36:22.866874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e360f3f-25ac-4580-83b3-5426f5a456c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:36:22.931073 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-7e360f3f-25ac-4580-83b3-5426f5a456c7 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:36:22.932269 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-7e360f3f-25ac-4580-83b3-5426f5a456c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:36:22.985975 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:36:23.030456 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 741/1489] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:36:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:36:23.070503 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5124c8-e771-4383-828d-6e7624f5f1d6 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:36:23.072067 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ec5124c8-e771-4383-828d-6e7624f5f1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:36:23.108288 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:36:23.174774 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5124c8-e771-4383-828d-6e7624f5f1d6 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:36:23.245888 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf172c68-446b-4238-8fb4-0d7901dfaaa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:36:23.246489 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf172c68-446b-4238-8fb4-0d7901dfaaa4 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:36:23.246761 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf172c68-446b-4238-8fb4-0d7901dfaaa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:36:23.247025 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf172c68-446b-4238-8fb4-0d7901dfaaa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:36:23.249127 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-cf172c68-446b-4238-8fb4-0d7901dfaaa4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:36:23.327308 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:36:23.355558 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf172c68-446b-4238-8fb4-0d7901dfaaa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:36:23.356991 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf172c68-446b-4238-8fb4-0d7901dfaaa4 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:36:23.374937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 749/1490] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:36:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:36:25.736946 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d035706-bad7-41b8-99e3-1d6ffdea007d None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:36:25.737520 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d035706-bad7-41b8-99e3-1d6ffdea007d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:36:25.737520 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d035706-bad7-41b8-99e3-1d6ffdea007d None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:36:25.738914 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 742/1491] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 08:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:36:25.744343 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c06b6-7439-4329-8c9e-141e9572843d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:36:25.744754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c06b6-7439-4329-8c9e-141e9572843d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:36:25.744897 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c06b6-7439-4329-8c9e-141e9572843d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:36:25.809358 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ce5c06b6-7439-4329-8c9e-141e9572843d None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:36:25.810557 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ce5c06b6-7439-4329-8c9e-141e9572843d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:36:25.905601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 750/1492] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 08:36:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:36:25.944242 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d035706-bad7-41b8-99e3-1d6ffdea007d None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:36:25.945832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-4d035706-bad7-41b8-99e3-1d6ffdea007d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:36:26.050428 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d035706-bad7-41b8-99e3-1d6ffdea007d None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:36:26.124798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7959401-fbd7-4120-85c0-09309ef1949e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:36:26.125440 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7959401-fbd7-4120-85c0-09309ef1949e service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:36:26.125723 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7959401-fbd7-4120-85c0-09309ef1949e service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:36:26.125997 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7959401-fbd7-4120-85c0-09309ef1949e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:36:26.127631 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c7959401-fbd7-4120-85c0-09309ef1949e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:36:26.136562 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:36:26.236446 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7959401-fbd7-4120-85c0-09309ef1949e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:36:26.237926 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7959401-fbd7-4120-85c0-09309ef1949e service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:36:26.255957 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 743/1493] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:36:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:36:35.903341 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=82 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:36:36.253792 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=50 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:38:05.044540 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1253a3e-724b-4b33-a503-458d87e4e595 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:38:05.045167 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1253a3e-724b-4b33-a503-458d87e4e595 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:38:05.045167 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1253a3e-724b-4b33-a503-458d87e4e595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:38:05.119064 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b1253a3e-724b-4b33-a503-458d87e4e595 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:38:05.120253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b1253a3e-724b-4b33-a503-458d87e4e595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:38:05.215130 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 751/1494] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:38:05.223651 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b04d6-990a-4a99-85cc-e53ec9ddc87a None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:38:05.224142 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b04d6-990a-4a99-85cc-e53ec9ddc87a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:38:05.224313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b04d6-990a-4a99-85cc-e53ec9ddc87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:38:05.301638 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-bf5b04d6-990a-4a99-85cc-e53ec9ddc87a None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:38:05.302765 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-bf5b04d6-990a-4a99-85cc-e53ec9ddc87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:38:05.398707 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 744/1495] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:38:05.403909 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1253a3e-724b-4b33-a503-458d87e4e595 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:38:05.405207 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b1253a3e-724b-4b33-a503-458d87e4e595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:38:05.416870 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1253a3e-724b-4b33-a503-458d87e4e595 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:38:05.477676 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:38:05.510469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb1207-193b-41df-acd5-89d96e5378bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:38:05.511309 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb1207-193b-41df-acd5-89d96e5378bf admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:38:05.511431 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb1207-193b-41df-acd5-89d96e5378bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:38:05.511647 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb1207-193b-41df-acd5-89d96e5378bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:38:05.512350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb1207-193b-41df-acd5-89d96e5378bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:38:05.512644 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb1207-193b-41df-acd5-89d96e5378bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:38:05.513690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: WARNING py.warnings [None req-7bcb1207-193b-41df-acd5-89d96e5378bf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 08:38:05.513690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: warnings.warn(msg) Aug 08 08:38:05.513690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: Aug 08 08:38:05.514575 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb1207-193b-41df-acd5-89d96e5378bf admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:38:05.526128 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 752/1496] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:38:05.535198 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b04d6-990a-4a99-85cc-e53ec9ddc87a None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:38:05.545381 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b04d6-990a-4a99-85cc-e53ec9ddc87a None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:38:05.613024 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:38:05.646645 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd46f1-ae86-4ab6-828f-48c50591e7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:38:05.647257 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd46f1-ae86-4ab6-828f-48c50591e7bd admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:38:05.647537 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd46f1-ae86-4ab6-828f-48c50591e7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:38:05.647809 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd46f1-ae86-4ab6-828f-48c50591e7bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:38:05.648443 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd46f1-ae86-4ab6-828f-48c50591e7bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:38:05.650201 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: WARNING py.warnings [None req-9cdd46f1-ae86-4ab6-828f-48c50591e7bd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 08:38:05.650201 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: warnings.warn(msg) Aug 08 08:38:05.650201 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: Aug 08 08:38:05.651384 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd46f1-ae86-4ab6-828f-48c50591e7bd admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:38:05.767303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 745/1497] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:38:05 2022] POST /identity/v3/projects => generated 379 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:38:05.771548 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb1207-193b-41df-acd5-89d96e5378bf admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:38:05.773054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7bcb1207-193b-41df-acd5-89d96e5378bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:38:05.806047 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb1207-193b-41df-acd5-89d96e5378bf admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:38:05.898397 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c169e96-8613-4228-a55c-70535ed4daf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:38:05.898998 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c169e96-8613-4228-a55c-70535ed4daf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:38:05.899277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c169e96-8613-4228-a55c-70535ed4daf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:38:05.899574 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c169e96-8613-4228-a55c-70535ed4daf1 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:38:05.900202 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c169e96-8613-4228-a55c-70535ed4daf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:38:05.901281 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: WARNING py.warnings [None req-7c169e96-8613-4228-a55c-70535ed4daf1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 08:38:05.901281 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: warnings.warn(msg) Aug 08 08:38:05.901281 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: Aug 08 08:38:05.903080 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c169e96-8613-4228-a55c-70535ed4daf1 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:38:06.025851 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 753/1498] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:38:05 2022] POST /identity/v3/users => generated 290 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:38:06.030726 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd46f1-ae86-4ab6-828f-48c50591e7bd admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:38:06.032201 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-9cdd46f1-ae86-4ab6-828f-48c50591e7bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:38:06.092956 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd46f1-ae86-4ab6-828f-48c50591e7bd admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:38:06.173522 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:38:06.183210 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301e481-44f8-4146-98a9-7127663b7902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:38:06.184352 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301e481-44f8-4146-98a9-7127663b7902 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:38:06.184628 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301e481-44f8-4146-98a9-7127663b7902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:38:06.184892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301e481-44f8-4146-98a9-7127663b7902 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:38:06.185557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b301e481-44f8-4146-98a9-7127663b7902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:38:06.189460 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: WARNING py.warnings [None req-b301e481-44f8-4146-98a9-7127663b7902 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 08:38:06.189460 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: warnings.warn(msg) Aug 08 08:38:06.189460 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: Aug 08 08:38:06.189460 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b301e481-44f8-4146-98a9-7127663b7902 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:38:06.201126 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 746/1499] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:38:06.205122 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c169e96-8613-4228-a55c-70535ed4daf1 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:38:06.206731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7c169e96-8613-4228-a55c-70535ed4daf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:38:06.218722 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c169e96-8613-4228-a55c-70535ed4daf1 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:38:06.243541 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:38:06.313985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dcf5c9-951e-4568-850f-837dc558c4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:38:06.314592 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcf5c9-951e-4568-850f-837dc558c4e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:38:06.314869 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcf5c9-951e-4568-850f-837dc558c4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:38:06.315140 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcf5c9-951e-4568-850f-837dc558c4e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:38:06.315773 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dcf5c9-951e-4568-850f-837dc558c4e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:38:06.316788 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: WARNING py.warnings [None req-26dcf5c9-951e-4568-850f-837dc558c4e4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 08:38:06.316788 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: warnings.warn(msg) Aug 08 08:38:06.316788 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: Aug 08 08:38:06.317706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dcf5c9-951e-4568-850f-837dc558c4e4 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:38:06.330202 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 754/1500] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:38:06.334460 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301e481-44f8-4146-98a9-7127663b7902 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:38:06.335958 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b301e481-44f8-4146-98a9-7127663b7902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:38:06.368632 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301e481-44f8-4146-98a9-7127663b7902 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:38:06.462185 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca0373-f5fe-438e-8d25-740887717681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:38:06.462721 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca0373-f5fe-438e-8d25-740887717681 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:38:06.462999 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca0373-f5fe-438e-8d25-740887717681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:38:06.463269 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca0373-f5fe-438e-8d25-740887717681 admin admin] PATH_INFO: `/v3/projects/52cca7930028432db9aa56fc2fdba470/users/fd7eec1d4ae14ac3b0c2c0e654cd41b9/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:38:06.485111 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebca0373-f5fe-438e-8d25-740887717681 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52cca7930028432db9aa56fc2fdba470, user_id=fd7eec1d4ae14ac3b0c2c0e654cd41b9, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:38:06.486160 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: WARNING py.warnings [None req-ebca0373-f5fe-438e-8d25-740887717681 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 08:38:06.486160 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: warnings.warn(msg) Aug 08 08:38:06.486160 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: Aug 08 08:38:06.488078 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebca0373-f5fe-438e-8d25-740887717681 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:38:06.634862 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 747/1501] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:38:06 2022] PUT /identity/v3/projects/52cca7930028432db9aa56fc2fdba470/users/fd7eec1d4ae14ac3b0c2c0e654cd41b9/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:38:06.647501 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1b937-4606-4d92-ab49-4c8f53d704c6 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:38:06.649236 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1b937-4606-4d92-ab49-4c8f53d704c6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:38:06.649236 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1b937-4606-4d92-ab49-4c8f53d704c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:38:06.703971 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-22f1b937-4606-4d92-ab49-4c8f53d704c6 None None] MFA Rules not processed for user `fd7eec1d4ae14ac3b0c2c0e654cd41b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:38:06.705169 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-22f1b937-4606-4d92-ab49-4c8f53d704c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:38:06.779550 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 755/1502] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:38:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:38:06.792669 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1481aa-2d66-49b5-82fc-a4d7e27e575c None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:38:06.793554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1481aa-2d66-49b5-82fc-a4d7e27e575c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:38:06.793836 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1481aa-2d66-49b5-82fc-a4d7e27e575c None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:38:06.795218 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 748/1503] 10.222.0.33 () {54 vars in 964 bytes} [Mon Aug 8 08:38:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:38:06.799904 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077a61f-782c-45d4-a087-3a6fc4dd9b10 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:38:06.800195 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077a61f-782c-45d4-a087-3a6fc4dd9b10 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:38:06.800470 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077a61f-782c-45d4-a087-3a6fc4dd9b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:38:06.830754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:38:06.875522 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b077a61f-782c-45d4-a087-3a6fc4dd9b10 None None] MFA Rules not processed for user `3262fa2d9c0d4d8998c9697994b48fef`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:38:06.876712 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b077a61f-782c-45d4-a087-3a6fc4dd9b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:38:06.951188 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 756/1504] 10.222.0.33 () {58 vars in 1061 bytes} [Mon Aug 8 08:38:06 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:38:06.958991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1481aa-2d66-49b5-82fc-a4d7e27e575c None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:38:06.960494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-9e1481aa-2d66-49b5-82fc-a4d7e27e575c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:38:07.009936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:38:07.062848 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1481aa-2d66-49b5-82fc-a4d7e27e575c None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:38:07.135211 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcc0374-45ba-47f6-a05f-ccc81f2c43b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:38:07.135819 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc0374-45ba-47f6-a05f-ccc81f2c43b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:38:07.136096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc0374-45ba-47f6-a05f-ccc81f2c43b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:38:07.136366 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc0374-45ba-47f6-a05f-ccc81f2c43b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:38:07.138032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8fcc0374-45ba-47f6-a05f-ccc81f2c43b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:38:07.237005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcc0374-45ba-47f6-a05f-ccc81f2c43b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:38:07.238486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcc0374-45ba-47f6-a05f-ccc81f2c43b2 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:38:07.241962 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:38:07.256198 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 749/1505] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:38:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:38:07.301658 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83351c3c-cfbc-4b5f-ac17-5aa88ca59f34 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:38:07.309659 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83351c3c-cfbc-4b5f-ac17-5aa88ca59f34 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:38:07.309659 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83351c3c-cfbc-4b5f-ac17-5aa88ca59f34 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:38:07.309659 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 757/1506] 10.222.0.33 () {52 vars in 830 bytes} [Mon Aug 8 08:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:38:07.309957 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcc0374-45ba-47f6-a05f-ccc81f2c43b2 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:38:07.309957 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8fcc0374-45ba-47f6-a05f-ccc81f2c43b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:38:07.406997 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcc0374-45ba-47f6-a05f-ccc81f2c43b2 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:38:07.502710 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df6df7-7b42-4c28-9284-6c66c5f9a9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:38:07.503343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df6df7-7b42-4c28-9284-6c66c5f9a9e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:38:07.503620 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df6df7-7b42-4c28-9284-6c66c5f9a9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:38:07.503888 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df6df7-7b42-4c28-9284-6c66c5f9a9e9 admin admin] PATH_INFO: `/v3/users/fd7eec1d4ae14ac3b0c2c0e654cd41b9` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:38:07.516440 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df6df7-7b42-4c28-9284-6c66c5f9a9e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd7eec1d4ae14ac3b0c2c0e654cd41b9)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:38:07.517473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: WARNING py.warnings [None req-84df6df7-7b42-4c28-9284-6c66c5f9a9e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 08:38:07.517473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: warnings.warn(msg) Aug 08 08:38:07.517473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: Aug 08 08:38:07.518663 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df6df7-7b42-4c28-9284-6c66c5f9a9e9 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:38:07.528653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:38:07.738236 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-84df6df7-7b42-4c28-9284-6c66c5f9a9e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd7eec1d4ae14ac3b0c2c0e654cd41b9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:38:07.738664 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-84df6df7-7b42-4c28-9284-6c66c5f9a9e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd7eec1d4ae14ac3b0c2c0e654cd41b9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:38:07.739179 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-84df6df7-7b42-4c28-9284-6c66c5f9a9e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd7eec1d4ae14ac3b0c2c0e654cd41b9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:38:07.753670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-84df6df7-7b42-4c28-9284-6c66c5f9a9e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd7eec1d4ae14ac3b0c2c0e654cd41b9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:38:07.825767 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-84df6df7-7b42-4c28-9284-6c66c5f9a9e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd7eec1d4ae14ac3b0c2c0e654cd41b9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:38:07.853791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 750/1507] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:38:07 2022] DELETE /identity/v3/users/fd7eec1d4ae14ac3b0c2c0e654cd41b9 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:38:07.858611 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac7ec86-c092-4330-9fee-f6d4b4db7dae None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:38:07.858900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac7ec86-c092-4330-9fee-f6d4b4db7dae None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:38:07.859174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac7ec86-c092-4330-9fee-f6d4b4db7dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:38:07.914431 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-5ac7ec86-c092-4330-9fee-f6d4b4db7dae None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:38:07.915620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5ac7ec86-c092-4330-9fee-f6d4b4db7dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:38:07.990887 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:38:08.003308 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 758/1508] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:38:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:38:08.012888 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df6df7-7b42-4c28-9284-6c66c5f9a9e9 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:38:08.014157 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-84df6df7-7b42-4c28-9284-6c66c5f9a9e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:38:08.116161 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df6df7-7b42-4c28-9284-6c66c5f9a9e9 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:38:08.170173 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:38:08.188885 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce331e90-2811-4064-812c-fcbc6fd0ce49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:38:08.189523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce331e90-2811-4064-812c-fcbc6fd0ce49 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:38:08.189797 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce331e90-2811-4064-812c-fcbc6fd0ce49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:38:08.190062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce331e90-2811-4064-812c-fcbc6fd0ce49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:38:08.191630 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ce331e90-2811-4064-812c-fcbc6fd0ce49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:38:08.297732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce331e90-2811-4064-812c-fcbc6fd0ce49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:38:08.299064 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce331e90-2811-4064-812c-fcbc6fd0ce49 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:38:08.317139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 751/1509] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:38:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:38:08.766628 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac7ec86-c092-4330-9fee-f6d4b4db7dae None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:38:08.766628 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5ac7ec86-c092-4330-9fee-f6d4b4db7dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:38:08.778631 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac7ec86-c092-4330-9fee-f6d4b4db7dae None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:38:08.872973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406f8ef-cb13-4188-b64b-fa01dda29222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:38:08.873632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406f8ef-cb13-4188-b64b-fa01dda29222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:38:08.873903 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406f8ef-cb13-4188-b64b-fa01dda29222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:38:08.874181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406f8ef-cb13-4188-b64b-fa01dda29222 admin admin] PATH_INFO: `/v3/projects/52cca7930028432db9aa56fc2fdba470` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:38:08.883912 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6406f8ef-cb13-4188-b64b-fa01dda29222 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52cca7930028432db9aa56fc2fdba470)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:38:08.884925 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: WARNING py.warnings [None req-6406f8ef-cb13-4188-b64b-fa01dda29222 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 08:38:08.884925 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: warnings.warn(msg) Aug 08 08:38:08.884925 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: Aug 08 08:38:08.886150 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6406f8ef-cb13-4188-b64b-fa01dda29222 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:38:09.002175 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6406f8ef-cb13-4188-b64b-fa01dda29222 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52cca7930028432db9aa56fc2fdba470', 'request_id': 'req-6406f8ef-cb13-4188-b64b-fa01dda29222'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:38:09.076882 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6406f8ef-cb13-4188-b64b-fa01dda29222 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52cca7930028432db9aa56fc2fdba470', 'request_id': 'req-6406f8ef-cb13-4188-b64b-fa01dda29222'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:38:09.090775 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6406f8ef-cb13-4188-b64b-fa01dda29222 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52cca7930028432db9aa56fc2fdba470', 'request_id': 'req-6406f8ef-cb13-4188-b64b-fa01dda29222'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:38:09.113388 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6406f8ef-cb13-4188-b64b-fa01dda29222 admin admin] The token cache is being invalidate because project 52cca7930028432db9aa56fc2fdba470 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:38:09.113673 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6406f8ef-cb13-4188-b64b-fa01dda29222 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:38:09.114094 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6406f8ef-cb13-4188-b64b-fa01dda29222 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:38:09.115488 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 759/1510] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:38:08 2022] DELETE /identity/v3/projects/52cca7930028432db9aa56fc2fdba470 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:38:18.314795 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=62 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:38:19.111538 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:39:32.785681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1b1a1-f252-42cb-962e-0b11a9062dd9 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:39:32.786282 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1b1a1-f252-42cb-962e-0b11a9062dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:39:32.786282 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1b1a1-f252-42cb-962e-0b11a9062dd9 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:39:32.787547 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 08:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:39:32.793050 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759369dc-9875-45a1-a743-71ec05eca8ab None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:39:32.796132 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759369dc-9875-45a1-a743-71ec05eca8ab None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:39:32.796132 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759369dc-9875-45a1-a743-71ec05eca8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:39:32.877056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-759369dc-9875-45a1-a743-71ec05eca8ab None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:39:32.878395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-759369dc-9875-45a1-a743-71ec05eca8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:39:32.991023 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 08:39:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:39:33.146475 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d1b1a1-f252-42cb-962e-0b11a9062dd9 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:39:33.148552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-66d1b1a1-f252-42cb-962e-0b11a9062dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:39:33.264960 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d1b1a1-f252-42cb-962e-0b11a9062dd9 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:39:33.270453 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad790fe4-9957-416c-9ff9-48c11351e987 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:39:33.271058 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad790fe4-9957-416c-9ff9-48c11351e987 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:39:33.271341 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad790fe4-9957-416c-9ff9-48c11351e987 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:39:33.271614 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad790fe4-9957-416c-9ff9-48c11351e987 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:39:33.280060 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad790fe4-9957-416c-9ff9-48c11351e987 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:39:33.281667 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad790fe4-9957-416c-9ff9-48c11351e987 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:39:33.290918 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 753/1513] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:39:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:39:34.119182 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e2e48-aedd-4a6f-87d4-1d96d081c06d None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:39:34.119722 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e2e48-aedd-4a6f-87d4-1d96d081c06d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:39:34.119722 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e2e48-aedd-4a6f-87d4-1d96d081c06d None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:39:34.121579 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 761/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 08:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:39:34.126425 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9974c5c-dd9a-4f5d-9755-25140760d689 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:39:34.126848 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9974c5c-dd9a-4f5d-9755-25140760d689 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:39:34.126973 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9974c5c-dd9a-4f5d-9755-25140760d689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:39:34.146459 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-f9974c5c-dd9a-4f5d-9755-25140760d689 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:39:34.147640 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f9974c5c-dd9a-4f5d-9755-25140760d689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:39:34.155483 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 754/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 08:39:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:39:34.309364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e2e48-aedd-4a6f-87d4-1d96d081c06d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:39:34.324451 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e2e48-aedd-4a6f-87d4-1d96d081c06d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:39:34.330013 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc6f8a-1931-413f-8ec3-381aa7072830 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:39:34.330606 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc6f8a-1931-413f-8ec3-381aa7072830 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:39:34.330885 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc6f8a-1931-413f-8ec3-381aa7072830 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:39:34.331157 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc6f8a-1931-413f-8ec3-381aa7072830 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:39:34.339729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdc6f8a-1931-413f-8ec3-381aa7072830 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:39:34.341393 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdc6f8a-1931-413f-8ec3-381aa7072830 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:39:34.350730 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 762/1516] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:39:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:39:34.989536 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4195776-5768-4742-99df-7d5d578b7a6e None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:39:34.989811 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4195776-5768-4742-99df-7d5d578b7a6e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:39:34.990081 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4195776-5768-4742-99df-7d5d578b7a6e None None] PATH_INFO: `/v3` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:39:34.991365 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 755/1517] 10.222.0.26 () {54 vars in 925 bytes} [Mon Aug 8 08:39:34 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:39:35.007181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025359c0-7794-45ac-9137-0097a7a4f835 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:39:35.007468 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025359c0-7794-45ac-9137-0097a7a4f835 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:39:35.007747 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025359c0-7794-45ac-9137-0097a7a4f835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:39:35.065834 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-025359c0-7794-45ac-9137-0097a7a4f835 None None] MFA Rules not processed for user `98c23a782b9b4144bace1248c3b2e754`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:39:35.067030 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-025359c0-7794-45ac-9137-0097a7a4f835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:39:35.083885 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:39:35.138896 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 763/1518] 10.222.0.26 () {58 vars in 1015 bytes} [Mon Aug 8 08:39:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:39:39.303261 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b140d-eebc-4792-b1ed-aeb81084adaf None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:39:39.303782 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b140d-eebc-4792-b1ed-aeb81084adaf None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:39:39.303782 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b140d-eebc-4792-b1ed-aeb81084adaf None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:39:39.305063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 756/1519] 10.222.0.33 () {54 vars in 961 bytes} [Mon Aug 8 08:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:39:39.310107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb989592-9ef7-44ff-8f8d-e756ead7ce7a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:39:39.310538 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb989592-9ef7-44ff-8f8d-e756ead7ce7a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:39:39.310638 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb989592-9ef7-44ff-8f8d-e756ead7ce7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:39:39.329684 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-cb989592-9ef7-44ff-8f8d-e756ead7ce7a None None] MFA Rules not processed for user `98c23a782b9b4144bace1248c3b2e754`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:39:39.330908 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-cb989592-9ef7-44ff-8f8d-e756ead7ce7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:39:39.339567 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 764/1520] 10.222.0.33 () {58 vars in 1058 bytes} [Mon Aug 8 08:39:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:39:39.347697 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b140d-eebc-4792-b1ed-aeb81084adaf None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:39:39.361300 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b140d-eebc-4792-b1ed-aeb81084adaf None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:39:39.366089 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0485f49-4a26-4dd6-9da4-628ec5b0b4ca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c23a782b9b4144bace1248c3b2e754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:39:39.366659 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0485f49-4a26-4dd6-9da4-628ec5b0b4ca service placement] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:39:39.366932 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0485f49-4a26-4dd6-9da4-628ec5b0b4ca service placement] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:39:39.367200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0485f49-4a26-4dd6-9da4-628ec5b0b4ca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:39:39.375376 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0485f49-4a26-4dd6-9da4-628ec5b0b4ca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:39:39.376926 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0485f49-4a26-4dd6-9da4-628ec5b0b4ca service placement] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:39:39.386315 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 757/1521] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:39:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:39:45.567473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4667d4-f916-46cf-809c-dd1bf16f639d None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:39:45.568039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4667d4-f916-46cf-809c-dd1bf16f639d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:39:45.568039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4667d4-f916-46cf-809c-dd1bf16f639d None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:39:45.569430 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 08:39:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:39:45.574382 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e781af-50b2-461e-8eb9-dbaaa194d548 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:39:45.574788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e781af-50b2-461e-8eb9-dbaaa194d548 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:39:45.575429 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e781af-50b2-461e-8eb9-dbaaa194d548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:39:45.595392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-59e781af-50b2-461e-8eb9-dbaaa194d548 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:39:45.596581 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-59e781af-50b2-461e-8eb9-dbaaa194d548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:39:45.604333 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 08:39:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:39:45.757483 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4667d4-f916-46cf-809c-dd1bf16f639d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:39:45.772144 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4667d4-f916-46cf-809c-dd1bf16f639d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:39:45.777526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b3cba-4a1a-4569-ae23-ce4c32fe24d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:39:45.778188 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b3cba-4a1a-4569-ae23-ce4c32fe24d4 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:39:45.778586 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b3cba-4a1a-4569-ae23-ce4c32fe24d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:39:45.778854 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b3cba-4a1a-4569-ae23-ce4c32fe24d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:39:45.786738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b3cba-4a1a-4569-ae23-ce4c32fe24d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:39:45.788313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b3cba-4a1a-4569-ae23-ce4c32fe24d4 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:39:45.797579 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 766/1524] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:39:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:39:46.610151 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd036a8-8f5d-4eb7-b81d-71a1d4248d00 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:39:46.610686 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd036a8-8f5d-4eb7-b81d-71a1d4248d00 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:39:46.610686 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd036a8-8f5d-4eb7-b81d-71a1d4248d00 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:39:46.612474 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 759/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 08:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:39:46.617199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82418731-635c-49c8-b0a5-1463ba928618 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:39:46.617634 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82418731-635c-49c8-b0a5-1463ba928618 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:39:46.617743 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82418731-635c-49c8-b0a5-1463ba928618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:39:46.637176 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-82418731-635c-49c8-b0a5-1463ba928618 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:39:46.638995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-82418731-635c-49c8-b0a5-1463ba928618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:39:46.646865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 767/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 08:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:39:46.798132 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd036a8-8f5d-4eb7-b81d-71a1d4248d00 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:39:46.812991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd036a8-8f5d-4eb7-b81d-71a1d4248d00 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:39:46.818271 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79249d-9cfc-4ecd-b0a5-14e220df369c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:39:46.818856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79249d-9cfc-4ecd-b0a5-14e220df369c service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:39:46.819132 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79249d-9cfc-4ecd-b0a5-14e220df369c service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:39:46.819400 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79249d-9cfc-4ecd-b0a5-14e220df369c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:39:46.827983 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79249d-9cfc-4ecd-b0a5-14e220df369c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:39:46.829736 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79249d-9cfc-4ecd-b0a5-14e220df369c service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:39:46.838805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 760/1527] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:39:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:39:50.661954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ee0f6-0f92-401d-9dc8-19170d3e529a None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:39:50.662488 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ee0f6-0f92-401d-9dc8-19170d3e529a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:39:50.662488 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ee0f6-0f92-401d-9dc8-19170d3e529a None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:39:50.663863 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 08:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:39:50.669543 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b99b96-ec19-4c4f-9751-6dcbb831cd95 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:39:50.669968 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b99b96-ec19-4c4f-9751-6dcbb831cd95 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:39:50.670093 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b99b96-ec19-4c4f-9751-6dcbb831cd95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:39:50.689705 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-a6b99b96-ec19-4c4f-9751-6dcbb831cd95 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:39:50.690895 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a6b99b96-ec19-4c4f-9751-6dcbb831cd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:39:50.698810 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 08:39:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:39:50.772500 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ee0f6-0f92-401d-9dc8-19170d3e529a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:39:50.774206 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-302ee0f6-0f92-401d-9dc8-19170d3e529a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:39:50.850957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ee0f6-0f92-401d-9dc8-19170d3e529a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:39:50.854529 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be75798f-c851-44a3-b320-46717d7fb583 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:39:50.855132 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75798f-c851-44a3-b320-46717d7fb583 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:39:50.855415 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75798f-c851-44a3-b320-46717d7fb583 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:39:50.855689 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75798f-c851-44a3-b320-46717d7fb583 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:39:50.865250 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be75798f-c851-44a3-b320-46717d7fb583 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:39:50.866672 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be75798f-c851-44a3-b320-46717d7fb583 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:39:50.876663 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 769/1530] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:39:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:39:52.873346 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78d0c5-6913-41a3-8cf4-99210c4c897d None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:39:52.873930 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78d0c5-6913-41a3-8cf4-99210c4c897d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:39:52.873930 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78d0c5-6913-41a3-8cf4-99210c4c897d None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:39:52.875310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 08:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 08:39:52.880526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cee15d-4538-4418-93d4-7b9424beec7b None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:39:52.880975 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cee15d-4538-4418-93d4-7b9424beec7b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:39:52.881095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cee15d-4538-4418-93d4-7b9424beec7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:39:52.900688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-21cee15d-4538-4418-93d4-7b9424beec7b None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:39:52.907718 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-21cee15d-4538-4418-93d4-7b9424beec7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:39:52.910593 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 08:39:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:39:52.945235 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78d0c5-6913-41a3-8cf4-99210c4c897d None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:39:52.956303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78d0c5-6913-41a3-8cf4-99210c4c897d None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:39:52.959845 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3a265a-c257-41e6-b26a-67274fa53158 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:39:52.960430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a265a-c257-41e6-b26a-67274fa53158 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:39:52.960709 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a265a-c257-41e6-b26a-67274fa53158 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:39:52.960982 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a265a-c257-41e6-b26a-67274fa53158 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:39:52.970790 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3a265a-c257-41e6-b26a-67274fa53158 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:39:52.972282 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3a265a-c257-41e6-b26a-67274fa53158 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:39:52.981863 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 763/1533] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:39:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:39:56.492314 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb75a9-8d53-4923-adf3-a079d6e0ed82 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:39:56.492860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb75a9-8d53-4923-adf3-a079d6e0ed82 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:39:56.492860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb75a9-8d53-4923-adf3-a079d6e0ed82 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:39:56.494392 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 08:39:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:39:56.500128 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5073d-6fff-455c-be93-c0d98bd71043 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:39:56.500577 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5073d-6fff-455c-be93-c0d98bd71043 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:39:56.500693 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5073d-6fff-455c-be93-c0d98bd71043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:39:56.519292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-7fd5073d-6fff-455c-be93-c0d98bd71043 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:39:56.520471 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-7fd5073d-6fff-455c-be93-c0d98bd71043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:39:56.528652 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 08:39:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:39:56.827947 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb75a9-8d53-4923-adf3-a079d6e0ed82 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:39:56.829647 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-84bb75a9-8d53-4923-adf3-a079d6e0ed82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:39:56.894018 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:39:56.907302 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb75a9-8d53-4923-adf3-a079d6e0ed82 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:39:56.910860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bce21d-87e3-4ee1-80af-427d72f2a09f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:39:56.911454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bce21d-87e3-4ee1-80af-427d72f2a09f service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:39:56.911730 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bce21d-87e3-4ee1-80af-427d72f2a09f service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:39:56.912000 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bce21d-87e3-4ee1-80af-427d72f2a09f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:39:56.920761 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bce21d-87e3-4ee1-80af-427d72f2a09f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:39:56.922271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bce21d-87e3-4ee1-80af-427d72f2a09f service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:39:56.931558 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 772/1536] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:39:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:39:58.928562 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1fedb-614b-4bd2-914b-67dd61734164 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:39:58.929166 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1fedb-614b-4bd2-914b-67dd61734164 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:39:58.929166 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1fedb-614b-4bd2-914b-67dd61734164 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:39:58.930454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 08:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:39:58.935603 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67e7d1-e5ed-484e-a3ab-5eb7eaa01b7b None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:39:58.936043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67e7d1-e5ed-484e-a3ab-5eb7eaa01b7b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:39:58.936147 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67e7d1-e5ed-484e-a3ab-5eb7eaa01b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:39:58.956448 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-7a67e7d1-e5ed-484e-a3ab-5eb7eaa01b7b None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:39:58.959908 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7a67e7d1-e5ed-484e-a3ab-5eb7eaa01b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:39:58.965569 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 08:39:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:39:59.265129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1fedb-614b-4bd2-914b-67dd61734164 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:39:59.266813 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c8d1fedb-614b-4bd2-914b-67dd61734164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:39:59.274467 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=84 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:39:59.280892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1fedb-614b-4bd2-914b-67dd61734164 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:39:59.284359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2f13b-ef3a-4794-9f3f-89fcc58823be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:39:59.284946 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2f13b-ef3a-4794-9f3f-89fcc58823be service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:39:59.288645 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2f13b-ef3a-4794-9f3f-89fcc58823be service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:39:59.288645 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2f13b-ef3a-4794-9f3f-89fcc58823be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:39:59.295107 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d2f13b-ef3a-4794-9f3f-89fcc58823be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:39:59.296604 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d2f13b-ef3a-4794-9f3f-89fcc58823be service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:39:59.306726 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 766/1539] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:39:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:40:08.951850 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=11 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:40:09.302897 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=5 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:23.998574 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0e8f4-c256-4596-8020-19801b135d53 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.000489 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0e8f4-c256-4596-8020-19801b135d53 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.001061 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0e8f4-c256-4596-8020-19801b135d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.041023 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-7dc0e8f4-c256-4596-8020-19801b135d53 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:24.042361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7dc0e8f4-c256-4596-8020-19801b135d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:24.054213 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 774/1540] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:24.064591 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90a6d9-559d-4832-8d01-1d9767cc56e2 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.065243 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90a6d9-559d-4832-8d01-1d9767cc56e2 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.065243 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90a6d9-559d-4832-8d01-1d9767cc56e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.095280 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-0d90a6d9-559d-4832-8d01-1d9767cc56e2 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:24.096843 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0d90a6d9-559d-4832-8d01-1d9767cc56e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:24.109305 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 767/1541] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:24.114292 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc0e8f4-c256-4596-8020-19801b135d53 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:24.116745 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e584cc7-5d7d-40b3-8aba-bcacc6379cd5 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.117028 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e584cc7-5d7d-40b3-8aba-bcacc6379cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.117359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e584cc7-5d7d-40b3-8aba-bcacc6379cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.131707 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc0e8f4-c256-4596-8020-19801b135d53 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:24.146919 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d418a3f-c5d4-4b65-a1e0-044da87a3af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:24.147592 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d418a3f-c5d4-4b65-a1e0-044da87a3af5 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.147722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-0e584cc7-5d7d-40b3-8aba-bcacc6379cd5 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:24.147817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d418a3f-c5d4-4b65-a1e0-044da87a3af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.148073 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d418a3f-c5d4-4b65-a1e0-044da87a3af5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.148731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d418a3f-c5d4-4b65-a1e0-044da87a3af5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:41:24.148857 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0e584cc7-5d7d-40b3-8aba-bcacc6379cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:24.149031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d418a3f-c5d4-4b65-a1e0-044da87a3af5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:24.151045 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d418a3f-c5d4-4b65-a1e0-044da87a3af5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:24.162951 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 768/1542] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 08 08:41:24.173200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dbe47f-65bd-425d-9af3-01dcdf56d888 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.173550 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dbe47f-65bd-425d-9af3-01dcdf56d888 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.173865 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dbe47f-65bd-425d-9af3-01dcdf56d888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.176322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 775/1543] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:41:24.184901 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d418a3f-c5d4-4b65-a1e0-044da87a3af5 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:24.203829 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-20dbe47f-65bd-425d-9af3-01dcdf56d888 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:24.205081 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-20dbe47f-65bd-425d-9af3-01dcdf56d888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:24.207845 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d418a3f-c5d4-4b65-a1e0-044da87a3af5 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:24.221429 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 769/1544] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:24.221829 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0fa7a1-30c8-4fd4-8b33-356dfcc5a719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:24.222453 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fa7a1-30c8-4fd4-8b33-356dfcc5a719 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.222735 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fa7a1-30c8-4fd4-8b33-356dfcc5a719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.223006 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fa7a1-30c8-4fd4-8b33-356dfcc5a719 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.223648 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0fa7a1-30c8-4fd4-8b33-356dfcc5a719 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:24.225542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: WARNING py.warnings [None req-9a0fa7a1-30c8-4fd4-8b33-356dfcc5a719 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 08:41:24.225542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: warnings.warn(msg) Aug 08 08:41:24.225542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: Aug 08 08:41:24.225860 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dbe47f-65bd-425d-9af3-01dcdf56d888 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:24.226753 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0fa7a1-30c8-4fd4-8b33-356dfcc5a719 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:24.236178 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dbe47f-65bd-425d-9af3-01dcdf56d888 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:24.251973 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295c28b-e238-4a71-a687-a8a9895c7c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:24.252323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295c28b-e238-4a71-a687-a8a9895c7c92 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.252598 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295c28b-e238-4a71-a687-a8a9895c7c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.257166 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295c28b-e238-4a71-a687-a8a9895c7c92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.257166 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295c28b-e238-4a71-a687-a8a9895c7c92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:41:24.257166 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295c28b-e238-4a71-a687-a8a9895c7c92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:24.257166 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: WARNING py.warnings [None req-9295c28b-e238-4a71-a687-a8a9895c7c92 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 08:41:24.257166 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: warnings.warn(msg) Aug 08 08:41:24.257166 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: Aug 08 08:41:24.257166 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295c28b-e238-4a71-a687-a8a9895c7c92 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:24.279997 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 770/1545] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:41:24.288031 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295c28b-e238-4a71-a687-a8a9895c7c92 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:24.300773 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295c28b-e238-4a71-a687-a8a9895c7c92 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:24.314876 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5463cf0-f471-45f0-9f26-a1cb84c9d1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:24.315491 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5463cf0-f471-45f0-9f26-a1cb84c9d1f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.315829 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5463cf0-f471-45f0-9f26-a1cb84c9d1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.316309 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5463cf0-f471-45f0-9f26-a1cb84c9d1f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.316938 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5463cf0-f471-45f0-9f26-a1cb84c9d1f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:24.318854 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5463cf0-f471-45f0-9f26-a1cb84c9d1f5 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:24.378613 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 776/1546] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:41:24 2022] POST /identity/v3/projects => generated 387 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:41:24.380280 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b10051-6052-4949-aaeb-a34494515fbe None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.380572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b10051-6052-4949-aaeb-a34494515fbe None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.380845 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b10051-6052-4949-aaeb-a34494515fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.418022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-01b10051-6052-4949-aaeb-a34494515fbe None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:24.419193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-01b10051-6052-4949-aaeb-a34494515fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:24.467428 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 771/1547] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:41:24 2022] POST /identity/v3/projects => generated 379 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:41:24.469159 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5463cf0-f471-45f0-9f26-a1cb84c9d1f5 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:24.513601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 777/1548] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:24.515362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519d0f7-3d4b-40a1-8e81-902138482fc4 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.515647 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519d0f7-3d4b-40a1-8e81-902138482fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.515918 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519d0f7-3d4b-40a1-8e81-902138482fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.543559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-5519d0f7-3d4b-40a1-8e81-902138482fc4 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:24.545106 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5519d0f7-3d4b-40a1-8e81-902138482fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:24.567548 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5463cf0-f471-45f0-9f26-a1cb84c9d1f5 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:24.572843 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46f374-cbf3-4787-bd97-ca4bfbaf5d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:24.573871 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46f374-cbf3-4787-bd97-ca4bfbaf5d0f admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.574148 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46f374-cbf3-4787-bd97-ca4bfbaf5d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.574420 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46f374-cbf3-4787-bd97-ca4bfbaf5d0f admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.575204 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46f374-cbf3-4787-bd97-ca4bfbaf5d0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:24.576211 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: WARNING py.warnings [None req-4b46f374-cbf3-4787-bd97-ca4bfbaf5d0f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 08:41:24.576211 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: warnings.warn(msg) Aug 08 08:41:24.576211 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: Aug 08 08:41:24.578043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46f374-cbf3-4787-bd97-ca4bfbaf5d0f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:24.610522 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 778/1549] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:24.611805 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5777705b-65b2-43db-bead-34c5cd8d0db6 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.612091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5777705b-65b2-43db-bead-34c5cd8d0db6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.612362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5777705b-65b2-43db-bead-34c5cd8d0db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.631472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-5777705b-65b2-43db-bead-34c5cd8d0db6 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:24.632694 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5777705b-65b2-43db-bead-34c5cd8d0db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:24.641050 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 779/1550] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:24.641827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5777705b-65b2-43db-bead-34c5cd8d0db6 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:24.651331 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5777705b-65b2-43db-bead-34c5cd8d0db6 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:24.653672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 772/1551] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:41:24 2022] POST /identity/v3/users => generated 294 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:41:24.654170 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a481dad-1e2f-4059-a595-f24f402022e8 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.654453 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a481dad-1e2f-4059-a595-f24f402022e8 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.654725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a481dad-1e2f-4059-a595-f24f402022e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.657342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9539c-ca29-446b-b430-4d9609fb98ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:24.657946 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9539c-ca29-446b-b430-4d9609fb98ca admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.658225 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9539c-ca29-446b-b430-4d9609fb98ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.658495 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9539c-ca29-446b-b430-4d9609fb98ca admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.659117 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade9539c-ca29-446b-b430-4d9609fb98ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:24.661028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade9539c-ca29-446b-b430-4d9609fb98ca admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:24.674980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-7a481dad-1e2f-4059-a595-f24f402022e8 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:24.676209 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-7a481dad-1e2f-4059-a595-f24f402022e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:24.684119 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 773/1552] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:24.691172 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90f149-d1c3-462c-bbb3-90b859104ebf None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.691172 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90f149-d1c3-462c-bbb3-90b859104ebf None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.691172 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90f149-d1c3-462c-bbb3-90b859104ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.705010 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-bd90f149-d1c3-462c-bbb3-90b859104ebf None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:24.706217 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-bd90f149-d1c3-462c-bbb3-90b859104ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:24.713986 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 774/1553] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:24.715107 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98d587-ae4b-442c-a6cb-6b150c3d1943 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.715384 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98d587-ae4b-442c-a6cb-6b150c3d1943 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.715649 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98d587-ae4b-442c-a6cb-6b150c3d1943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.730828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 780/1554] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:41:24 2022] POST /identity/v3/users => generated 290 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:41:24.743523 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d572d-34b1-4278-a477-4ee5d1113892 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.743523 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d572d-34b1-4278-a477-4ee5d1113892 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.743523 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d572d-34b1-4278-a477-4ee5d1113892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.743815 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-9d98d587-ae4b-442c-a6cb-6b150c3d1943 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:24.743815 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-9d98d587-ae4b-442c-a6cb-6b150c3d1943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:24.743969 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 775/1555] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:24.745093 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67544098-294d-410c-b254-1f96749b9d93 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.745438 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67544098-294d-410c-b254-1f96749b9d93 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.745971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67544098-294d-410c-b254-1f96749b9d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.757578 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8c4d572d-34b1-4278-a477-4ee5d1113892 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:24.758773 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8c4d572d-34b1-4278-a477-4ee5d1113892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:24.764376 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-67544098-294d-410c-b254-1f96749b9d93 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:24.765962 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-67544098-294d-410c-b254-1f96749b9d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:24.767766 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 781/1556] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:24.770530 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be32a88-d09d-4e43-97be-0b91fc5175a7 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.771053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be32a88-d09d-4e43-97be-0b91fc5175a7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.771363 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be32a88-d09d-4e43-97be-0b91fc5175a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.777166 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 776/1557] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:24.777166 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67544098-294d-410c-b254-1f96749b9d93 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:24.787167 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67544098-294d-410c-b254-1f96749b9d93 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:24.791513 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4be32a88-d09d-4e43-97be-0b91fc5175a7 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:24.791701 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f7bd8-5aa7-4982-bd59-85b477a8555f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:24.792296 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f7bd8-5aa7-4982-bd59-85b477a8555f admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.792571 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f7bd8-5aa7-4982-bd59-85b477a8555f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.792687 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4be32a88-d09d-4e43-97be-0b91fc5175a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:24.792840 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f7bd8-5aa7-4982-bd59-85b477a8555f admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.793506 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f7bd8-5aa7-4982-bd59-85b477a8555f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:24.795079 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f7bd8-5aa7-4982-bd59-85b477a8555f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:24.800932 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 782/1558] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:24.802339 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193864e-841d-4a8d-ba47-67e5993a5daa None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.802618 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193864e-841d-4a8d-ba47-67e5993a5daa None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.802887 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193864e-841d-4a8d-ba47-67e5993a5daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.808317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 777/1559] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:41:24.809064 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f7bd8-5aa7-4982-bd59-85b477a8555f admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:24.818864 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f7bd8-5aa7-4982-bd59-85b477a8555f admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:24.823727 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f5edf-30e7-4e46-bab8-b5879b100624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:24.824349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f5edf-30e7-4e46-bab8-b5879b100624 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.824468 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-3193864e-841d-4a8d-ba47-67e5993a5daa None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:24.824630 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f5edf-30e7-4e46-bab8-b5879b100624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.824903 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f5edf-30e7-4e46-bab8-b5879b100624 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.825569 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3193864e-841d-4a8d-ba47-67e5993a5daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:24.825682 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f5edf-30e7-4e46-bab8-b5879b100624 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:41:24.825872 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f5edf-30e7-4e46-bab8-b5879b100624 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:24.828024 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f5edf-30e7-4e46-bab8-b5879b100624 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:24.834162 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 783/1560] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:24.834957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193864e-841d-4a8d-ba47-67e5993a5daa None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:24.840589 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 778/1561] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:41:24.841722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d553fb-c867-49c4-9553-6d2bf23cac77 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.842011 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d553fb-c867-49c4-9553-6d2bf23cac77 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.842287 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d553fb-c867-49c4-9553-6d2bf23cac77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.852109 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193864e-841d-4a8d-ba47-67e5993a5daa None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:24.857798 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1436d1-e560-42c2-896a-444bb29b11ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:24.859134 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1436d1-e560-42c2-896a-444bb29b11ab admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.859511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1436d1-e560-42c2-896a-444bb29b11ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.859783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1436d1-e560-42c2-896a-444bb29b11ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.860424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1436d1-e560-42c2-896a-444bb29b11ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:24.862045 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1436d1-e560-42c2-896a-444bb29b11ab admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:24.863470 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:24.865095 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-23d553fb-c867-49c4-9553-6d2bf23cac77 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:24.866311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-23d553fb-c867-49c4-9553-6d2bf23cac77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:24.872691 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 784/1562] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:41:24.874017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef8589-de72-4fd1-bdf5-1387bce92fa2 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.878288 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef8589-de72-4fd1-bdf5-1387bce92fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.878418 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 779/1563] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:24.878514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef8589-de72-4fd1-bdf5-1387bce92fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.878622 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d553fb-c867-49c4-9553-6d2bf23cac77 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:24.887614 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d553fb-c867-49c4-9553-6d2bf23cac77 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:24.893024 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038dae4-f046-4e6d-aa77-3cfc5dbd6f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:24.893656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038dae4-f046-4e6d-aa77-3cfc5dbd6f61 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.893935 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038dae4-f046-4e6d-aa77-3cfc5dbd6f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.894206 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038dae4-f046-4e6d-aa77-3cfc5dbd6f61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.894846 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6038dae4-f046-4e6d-aa77-3cfc5dbd6f61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:41:24.895146 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6038dae4-f046-4e6d-aa77-3cfc5dbd6f61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:24.895275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-abef8589-de72-4fd1-bdf5-1387bce92fa2 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:24.896718 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6038dae4-f046-4e6d-aa77-3cfc5dbd6f61 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:24.897470 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-abef8589-de72-4fd1-bdf5-1387bce92fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:24.905723 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 785/1564] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:24.906309 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef8589-de72-4fd1-bdf5-1387bce92fa2 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:24.912055 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 780/1565] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:41:24.912872 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038dae4-f046-4e6d-aa77-3cfc5dbd6f61 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:24.916448 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef8589-de72-4fd1-bdf5-1387bce92fa2 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:24.921656 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b6a9b-7094-4e6a-90c9-45fd59861b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:24.922262 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b6a9b-7094-4e6a-90c9-45fd59861b8b admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.922540 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b6a9b-7094-4e6a-90c9-45fd59861b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.922808 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b6a9b-7094-4e6a-90c9-45fd59861b8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.923312 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038dae4-f046-4e6d-aa77-3cfc5dbd6f61 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:24.923494 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b6a9b-7094-4e6a-90c9-45fd59861b8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:41:24.923781 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b6a9b-7094-4e6a-90c9-45fd59861b8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:24.925402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b6a9b-7094-4e6a-90c9-45fd59861b8b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:24.932274 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fec4c2-50e1-4547-9d0a-6784b487dd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:24.932944 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fec4c2-50e1-4547-9d0a-6784b487dd99 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.933259 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fec4c2-50e1-4547-9d0a-6784b487dd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.933549 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fec4c2-50e1-4547-9d0a-6784b487dd99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.934183 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fec4c2-50e1-4547-9d0a-6784b487dd99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:24.935826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fec4c2-50e1-4547-9d0a-6784b487dd99 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:24.937816 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 786/1566] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:41:24.938633 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b6a9b-7094-4e6a-90c9-45fd59861b8b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:24.948299 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b6a9b-7094-4e6a-90c9-45fd59861b8b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:24.955840 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 781/1567] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:41:24.956003 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e172f-adfa-4164-a7d1-7038f88183ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:24.956131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fec4c2-50e1-4547-9d0a-6784b487dd99 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:24.956236 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e172f-adfa-4164-a7d1-7038f88183ae admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.956236 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e172f-adfa-4164-a7d1-7038f88183ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.956236 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e172f-adfa-4164-a7d1-7038f88183ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.956236 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e172f-adfa-4164-a7d1-7038f88183ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:41:24.956236 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e172f-adfa-4164-a7d1-7038f88183ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:24.957340 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e172f-adfa-4164-a7d1-7038f88183ae admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:24.963652 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fec4c2-50e1-4547-9d0a-6784b487dd99 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:24.968522 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359fb94-fbf9-4143-b92d-205319442b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:24.969117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359fb94-fbf9-4143-b92d-205319442b81 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.969427 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359fb94-fbf9-4143-b92d-205319442b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.969704 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359fb94-fbf9-4143-b92d-205319442b81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.969819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 787/1568] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:41:24.970325 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f359fb94-fbf9-4143-b92d-205319442b81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:24.971145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e172f-adfa-4164-a7d1-7038f88183ae admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:24.972565 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f359fb94-fbf9-4143-b92d-205319442b81 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:24.982073 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e172f-adfa-4164-a7d1-7038f88183ae admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:24.985188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:24.988486 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb92ed1-878f-4379-a01b-5a775b1aeca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:24.989083 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb92ed1-878f-4379-a01b-5a775b1aeca2 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:24.989394 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb92ed1-878f-4379-a01b-5a775b1aeca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:24.989664 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb92ed1-878f-4379-a01b-5a775b1aeca2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:24.990302 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb92ed1-878f-4379-a01b-5a775b1aeca2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:24.991883 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb92ed1-878f-4379-a01b-5a775b1aeca2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:25.002743 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 788/1569] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:41:25.003668 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb92ed1-878f-4379-a01b-5a775b1aeca2 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:25.013373 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb92ed1-878f-4379-a01b-5a775b1aeca2 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:25.018167 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb293b-0c42-4c00-92d0-6f4921270a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:25.018804 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb293b-0c42-4c00-92d0-6f4921270a28 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:25.019095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb293b-0c42-4c00-92d0-6f4921270a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:25.019390 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb293b-0c42-4c00-92d0-6f4921270a28 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:25.020016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cb293b-0c42-4c00-92d0-6f4921270a28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:41:25.020312 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cb293b-0c42-4c00-92d0-6f4921270a28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:25.021965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cb293b-0c42-4c00-92d0-6f4921270a28 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:25.031024 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 782/1570] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:41:24 2022] POST /identity/v3/projects => generated 387 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:41:25.034023 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 789/1571] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:41:25.045436 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359fb94-fbf9-4143-b92d-205319442b81 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:25.045599 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb293b-0c42-4c00-92d0-6f4921270a28 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:25.045712 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb293b-0c42-4c00-92d0-6f4921270a28 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:25.103573 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409c43d-c9ac-43d6-b510-b5bffd08a9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:25.104176 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409c43d-c9ac-43d6-b510-b5bffd08a9c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:25.104450 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409c43d-c9ac-43d6-b510-b5bffd08a9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:25.104742 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409c43d-c9ac-43d6-b510-b5bffd08a9c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:25.105277 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359fb94-fbf9-4143-b92d-205319442b81 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:25.105766 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7409c43d-c9ac-43d6-b510-b5bffd08a9c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:25.107260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7409c43d-c9ac-43d6-b510-b5bffd08a9c5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:25.109943 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c08ee-f249-4b15-a8f6-b859175ca6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:25.110546 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c08ee-f249-4b15-a8f6-b859175ca6d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:25.110820 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c08ee-f249-4b15-a8f6-b859175ca6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:25.111087 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c08ee-f249-4b15-a8f6-b859175ca6d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:25.111723 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c08ee-f249-4b15-a8f6-b859175ca6d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:41:25.112017 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c08ee-f249-4b15-a8f6-b859175ca6d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:25.113636 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c08ee-f249-4b15-a8f6-b859175ca6d6 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:25.125852 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 783/1572] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:41:25.126649 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c08ee-f249-4b15-a8f6-b859175ca6d6 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:25.136181 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c08ee-f249-4b15-a8f6-b859175ca6d6 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:25.141225 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9d500-8645-4595-8af4-9eb528403072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:25.141798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9d500-8645-4595-8af4-9eb528403072 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:25.142114 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9d500-8645-4595-8af4-9eb528403072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:25.142389 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9d500-8645-4595-8af4-9eb528403072 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:25.142988 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af9d500-8645-4595-8af4-9eb528403072 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:25.144850 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af9d500-8645-4595-8af4-9eb528403072 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:25.191415 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 790/1573] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:41:25 2022] POST /identity/v3/projects => generated 403 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:41:25.192414 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409c43d-c9ac-43d6-b510-b5bffd08a9c5 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:25.208375 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 784/1574] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:41:25 2022] POST /identity/v3/projects => generated 377 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:41:25.209344 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9d500-8645-4595-8af4-9eb528403072 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:25.256008 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409c43d-c9ac-43d6-b510-b5bffd08a9c5 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:25.271335 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9d500-8645-4595-8af4-9eb528403072 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:25.276258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1a82d-a92a-4dd7-90ef-0a41d74c4d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:25.276860 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1a82d-a92a-4dd7-90ef-0a41d74c4d22 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:25.277157 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1a82d-a92a-4dd7-90ef-0a41d74c4d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:25.277443 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1a82d-a92a-4dd7-90ef-0a41d74c4d22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:25.278061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f1a82d-a92a-4dd7-90ef-0a41d74c4d22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:25.279998 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f1a82d-a92a-4dd7-90ef-0a41d74c4d22 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:25.284380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:25.311002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24130a9-00e3-4508-bc03-4374ef259e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:25.312159 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24130a9-00e3-4508-bc03-4374ef259e8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:25.312437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24130a9-00e3-4508-bc03-4374ef259e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:25.312855 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24130a9-00e3-4508-bc03-4374ef259e8d admin admin] PATH_INFO: `/v3/projects/2841a629bc1f4885a356218c2d0721b7/users/b3e5a78c3b6c48d9a4aed7d36fd54736/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:25.326259 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24130a9-00e3-4508-bc03-4374ef259e8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2841a629bc1f4885a356218c2d0721b7, user_id=b3e5a78c3b6c48d9a4aed7d36fd54736, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:25.328126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: WARNING py.warnings [None req-f24130a9-00e3-4508-bc03-4374ef259e8d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 08:41:25.328126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: warnings.warn(msg) Aug 08 08:41:25.328126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: Aug 08 08:41:25.330102 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24130a9-00e3-4508-bc03-4374ef259e8d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:25.350582 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 785/1575] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:41:25 2022] POST /identity/v3/projects => generated 381 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:41:25.351533 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1a82d-a92a-4dd7-90ef-0a41d74c4d22 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:25.399295 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:25.412918 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1a82d-a92a-4dd7-90ef-0a41d74c4d22 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:25.417629 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a3ee5-4f3e-4ea7-aad6-493832f04a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:25.418308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a3ee5-4f3e-4ea7-aad6-493832f04a93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:25.418584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a3ee5-4f3e-4ea7-aad6-493832f04a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:25.418872 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a3ee5-4f3e-4ea7-aad6-493832f04a93 admin admin] PATH_INFO: `/v3/projects/cee46083a26948a9a4aba2068f5c816c/users/489aa74b7ab34d5789416739b8751473/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:25.433484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a3ee5-4f3e-4ea7-aad6-493832f04a93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cee46083a26948a9a4aba2068f5c816c, user_id=489aa74b7ab34d5789416739b8751473, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:25.435663 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 791/1576] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:41:25 2022] PUT /identity/v3/projects/2841a629bc1f4885a356218c2d0721b7/users/b3e5a78c3b6c48d9a4aed7d36fd54736/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:41:25.436122 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a3ee5-4f3e-4ea7-aad6-493832f04a93 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:25.436333 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24130a9-00e3-4508-bc03-4374ef259e8d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:25.497395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24130a9-00e3-4508-bc03-4374ef259e8d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:25.503863 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 786/1577] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:41:25 2022] PUT /identity/v3/projects/cee46083a26948a9a4aba2068f5c816c/users/489aa74b7ab34d5789416739b8751473/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:41:25.506920 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a3ee5-4f3e-4ea7-aad6-493832f04a93 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:25.553826 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf1176-c55b-492f-934d-56a5a8b54f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:25.555235 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf1176-c55b-492f-934d-56a5a8b54f8f admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:25.555235 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf1176-c55b-492f-934d-56a5a8b54f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:25.555235 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf1176-c55b-492f-934d-56a5a8b54f8f admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:25.555531 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf1176-c55b-492f-934d-56a5a8b54f8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:25.557452 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf1176-c55b-492f-934d-56a5a8b54f8f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:25.582194 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a3ee5-4f3e-4ea7-aad6-493832f04a93 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:25.586897 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3164e2-9671-4b5b-9561-88a61e09fa0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:25.587529 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3164e2-9671-4b5b-9561-88a61e09fa0c admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:25.587807 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3164e2-9671-4b5b-9561-88a61e09fa0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:25.588074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3164e2-9671-4b5b-9561-88a61e09fa0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:25.588807 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3164e2-9671-4b5b-9561-88a61e09fa0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:25.590718 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3164e2-9671-4b5b-9561-88a61e09fa0c admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:25.672908 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 792/1578] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:41:25 2022] POST /identity/v3/users => generated 294 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:41:25.672908 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf1176-c55b-492f-934d-56a5a8b54f8f admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:25.680340 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf1176-c55b-492f-934d-56a5a8b54f8f admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:25.685167 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b2c0a-09d4-4822-b65b-cc1062121d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:25.686519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b2c0a-09d4-4822-b65b-cc1062121d70 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:25.686519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b2c0a-09d4-4822-b65b-cc1062121d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:25.686519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b2c0a-09d4-4822-b65b-cc1062121d70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:25.687607 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b2c0a-09d4-4822-b65b-cc1062121d70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:25.691480 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 787/1579] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:41:25 2022] POST /identity/v3/projects => generated 385 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:41:25.691480 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3164e2-9671-4b5b-9561-88a61e09fa0c admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:25.691729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b2c0a-09d4-4822-b65b-cc1062121d70 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:25.751207 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3164e2-9671-4b5b-9561-88a61e09fa0c admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:25.755929 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eed285e-8cb4-4926-94fb-10010a900c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:25.756522 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed285e-8cb4-4926-94fb-10010a900c87 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:25.756796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed285e-8cb4-4926-94fb-10010a900c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:25.757063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed285e-8cb4-4926-94fb-10010a900c87 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:25.757755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eed285e-8cb4-4926-94fb-10010a900c87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:25.759700 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eed285e-8cb4-4926-94fb-10010a900c87 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:25.771847 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 793/1580] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:41:25 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:41:25.771847 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b2c0a-09d4-4822-b65b-cc1062121d70 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:25.800737 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:25.823734 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 788/1581] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:41:25 2022] POST /identity/v3/users => generated 302 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:41:25.824573 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eed285e-8cb4-4926-94fb-10010a900c87 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:25.833134 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b2c0a-09d4-4822-b65b-cc1062121d70 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:25.834406 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eed285e-8cb4-4926-94fb-10010a900c87 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:25.837918 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b0cc3-92ea-4d82-9d3e-3da366c34063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:25.838547 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b0cc3-92ea-4d82-9d3e-3da366c34063 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:25.838794 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b0cc3-92ea-4d82-9d3e-3da366c34063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:25.838910 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd28678-c4e4-4192-ae6a-48bf428d9ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:25.839058 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b0cc3-92ea-4d82-9d3e-3da366c34063 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:25.839498 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd28678-c4e4-4192-ae6a-48bf428d9ddd admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:25.839675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0b0cc3-92ea-4d82-9d3e-3da366c34063 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:25.839799 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd28678-c4e4-4192-ae6a-48bf428d9ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:25.840037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd28678-c4e4-4192-ae6a-48bf428d9ddd admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:25.840672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd28678-c4e4-4192-ae6a-48bf428d9ddd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:25.841721 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0b0cc3-92ea-4d82-9d3e-3da366c34063 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:25.842559 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd28678-c4e4-4192-ae6a-48bf428d9ddd admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:25.913750 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 794/1582] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:41:25 2022] POST /identity/v3/users => generated 289 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:41:25.914979 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab31fb3-455c-43bc-828d-9c347483426a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:25.915266 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab31fb3-455c-43bc-828d-9c347483426a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:25.915539 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab31fb3-455c-43bc-828d-9c347483426a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:25.934484 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-9ab31fb3-455c-43bc-828d-9c347483426a None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:25.935683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9ab31fb3-455c-43bc-828d-9c347483426a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:25.943923 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 789/1583] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:41:25 2022] POST /identity/v3/users => generated 291 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:41:25.948045 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 795/1584] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:25.948156 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e938c5-215b-4c7d-9b89-b30544b74808 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:25.948250 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab31fb3-455c-43bc-828d-9c347483426a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:25.948330 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e938c5-215b-4c7d-9b89-b30544b74808 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:25.948330 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e938c5-215b-4c7d-9b89-b30544b74808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:25.953970 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab31fb3-455c-43bc-828d-9c347483426a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:25.959847 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758efda-4b56-4e0c-9f04-d76bdfb88155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:25.960485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758efda-4b56-4e0c-9f04-d76bdfb88155 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:25.960765 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758efda-4b56-4e0c-9f04-d76bdfb88155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:25.961031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758efda-4b56-4e0c-9f04-d76bdfb88155 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:25.961706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5758efda-4b56-4e0c-9f04-d76bdfb88155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:25.963301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5758efda-4b56-4e0c-9f04-d76bdfb88155 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:25.967218 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-e9e938c5-215b-4c7d-9b89-b30544b74808 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:25.969088 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e9e938c5-215b-4c7d-9b89-b30544b74808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:25.977712 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 796/1585] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:41:25.977712 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758efda-4b56-4e0c-9f04-d76bdfb88155 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:25.978005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 790/1586] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:25.978648 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e938c5-215b-4c7d-9b89-b30544b74808 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:25.989019 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e938c5-215b-4c7d-9b89-b30544b74808 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:25.989229 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758efda-4b56-4e0c-9f04-d76bdfb88155 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:25.993580 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6003f7-ad6e-4bca-83c9-65d5293829ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:25.993728 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc7b3b-aa8c-4b7d-9d33-94f137aee4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:25.994176 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6003f7-ad6e-4bca-83c9-65d5293829ef admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:25.994275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc7b3b-aa8c-4b7d-9d33-94f137aee4fa admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:25.994449 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6003f7-ad6e-4bca-83c9-65d5293829ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:25.994556 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc7b3b-aa8c-4b7d-9d33-94f137aee4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:25.994713 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6003f7-ad6e-4bca-83c9-65d5293829ef admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:25.994809 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc7b3b-aa8c-4b7d-9d33-94f137aee4fa admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:25.995323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6003f7-ad6e-4bca-83c9-65d5293829ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:25.995437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc7b3b-aa8c-4b7d-9d33-94f137aee4fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:25.997236 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6003f7-ad6e-4bca-83c9-65d5293829ef admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:25.997359 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc7b3b-aa8c-4b7d-9d33-94f137aee4fa admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:26.097847 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 791/1587] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:41:25 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:41:26.098557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6003f7-ad6e-4bca-83c9-65d5293829ef admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:26.104639 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:26.107943 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6003f7-ad6e-4bca-83c9-65d5293829ef admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:26.108590 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 797/1588] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:41:25 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:41:26.110562 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc7b3b-aa8c-4b7d-9d33-94f137aee4fa admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:26.114943 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466118ad-0537-43a3-93b8-4e9abd7634f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:26.115537 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466118ad-0537-43a3-93b8-4e9abd7634f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:26.115813 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466118ad-0537-43a3-93b8-4e9abd7634f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:26.116088 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466118ad-0537-43a3-93b8-4e9abd7634f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:26.116724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466118ad-0537-43a3-93b8-4e9abd7634f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:26.118407 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466118ad-0537-43a3-93b8-4e9abd7634f8 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:26.121491 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc7b3b-aa8c-4b7d-9d33-94f137aee4fa admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:26.126044 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e60948-0da0-4dcd-a46e-eb01baf50b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:26.126641 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e60948-0da0-4dcd-a46e-eb01baf50b5c admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:26.126921 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e60948-0da0-4dcd-a46e-eb01baf50b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:26.127192 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e60948-0da0-4dcd-a46e-eb01baf50b5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:26.127832 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e60948-0da0-4dcd-a46e-eb01baf50b5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:26.128959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 792/1589] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:41:26.129520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e60948-0da0-4dcd-a46e-eb01baf50b5c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:26.129776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466118ad-0537-43a3-93b8-4e9abd7634f8 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:26.138976 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466118ad-0537-43a3-93b8-4e9abd7634f8 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:26.142852 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 798/1590] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:41:26.142852 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e60948-0da0-4dcd-a46e-eb01baf50b5c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:26.146971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f3bcce-2818-437f-b8ff-80f409bfbe01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:26.146971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3bcce-2818-437f-b8ff-80f409bfbe01 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:26.146971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3bcce-2818-437f-b8ff-80f409bfbe01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:26.146971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3bcce-2818-437f-b8ff-80f409bfbe01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:26.147372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f3bcce-2818-437f-b8ff-80f409bfbe01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:26.147560 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f3bcce-2818-437f-b8ff-80f409bfbe01 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:26.158981 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 793/1591] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:41:26.160004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f3bcce-2818-437f-b8ff-80f409bfbe01 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:26.170474 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f3bcce-2818-437f-b8ff-80f409bfbe01 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:26.176172 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b64c45-ad94-4efb-9891-d2b5ddef7cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:26.176766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b64c45-ad94-4efb-9891-d2b5ddef7cdf admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:26.177042 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b64c45-ad94-4efb-9891-d2b5ddef7cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:26.177336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b64c45-ad94-4efb-9891-d2b5ddef7cdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:26.177971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b64c45-ad94-4efb-9891-d2b5ddef7cdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:26.179538 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b64c45-ad94-4efb-9891-d2b5ddef7cdf admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:26.189780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 794/1592] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:41:26.190580 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b64c45-ad94-4efb-9891-d2b5ddef7cdf admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:26.195060 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e4b64c45-ad94-4efb-9891-d2b5ddef7cdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:26.201261 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e60948-0da0-4dcd-a46e-eb01baf50b5c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:26.205494 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea3968-0796-4cc1-81cb-650c2311e573 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:26.206100 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea3968-0796-4cc1-81cb-650c2311e573 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:26.206378 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea3968-0796-4cc1-81cb-650c2311e573 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:26.206648 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea3968-0796-4cc1-81cb-650c2311e573 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:26.209159 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b64c45-ad94-4efb-9891-d2b5ddef7cdf admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:26.212731 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dfac3e-fe37-4e9c-95e1-00c35259bac4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:26.213391 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfac3e-fe37-4e9c-95e1-00c35259bac4 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:26.213672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfac3e-fe37-4e9c-95e1-00c35259bac4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:26.213949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfac3e-fe37-4e9c-95e1-00c35259bac4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:26.215725 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea3968-0796-4cc1-81cb-650c2311e573 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:26.217160 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea3968-0796-4cc1-81cb-650c2311e573 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:26.223813 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dfac3e-fe37-4e9c-95e1-00c35259bac4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:26.225312 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dfac3e-fe37-4e9c-95e1-00c35259bac4 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:26.227612 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 799/1593] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:41:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:41:26.228488 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea3968-0796-4cc1-81cb-650c2311e573 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:26.235320 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 795/1594] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:41:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:41:26.235961 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dfac3e-fe37-4e9c-95e1-00c35259bac4 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:26.239662 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea3968-0796-4cc1-81cb-650c2311e573 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:26.246327 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122fd0aa-461d-4fd2-a704-f58e771a5d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:26.246903 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dfac3e-fe37-4e9c-95e1-00c35259bac4 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:26.247709 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fd0aa-461d-4fd2-a704-f58e771a5d1f admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:26.248192 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fd0aa-461d-4fd2-a704-f58e771a5d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:26.248663 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fd0aa-461d-4fd2-a704-f58e771a5d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:26.250050 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122fd0aa-461d-4fd2-a704-f58e771a5d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:26.252044 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122fd0aa-461d-4fd2-a704-f58e771a5d1f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:26.252160 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e32823-171d-4514-b670-f69c48d4d52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:26.252696 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e32823-171d-4514-b670-f69c48d4d52f admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:26.252979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e32823-171d-4514-b670-f69c48d4d52f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:26.253289 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e32823-171d-4514-b670-f69c48d4d52f admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:26.253935 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e32823-171d-4514-b670-f69c48d4d52f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:26.255533 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e32823-171d-4514-b670-f69c48d4d52f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:26.264364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 800/1595] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:41:26.265769 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122fd0aa-461d-4fd2-a704-f58e771a5d1f admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:26.274077 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 796/1596] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:41:26.274077 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e32823-171d-4514-b670-f69c48d4d52f admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:26.275462 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122fd0aa-461d-4fd2-a704-f58e771a5d1f admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:26.282379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df07c2-0056-4b05-bbea-e6e38c49932c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:26.282760 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e32823-171d-4514-b670-f69c48d4d52f admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:26.282979 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df07c2-0056-4b05-bbea-e6e38c49932c admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:26.283253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df07c2-0056-4b05-bbea-e6e38c49932c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:26.283557 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df07c2-0056-4b05-bbea-e6e38c49932c admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:26.284193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df07c2-0056-4b05-bbea-e6e38c49932c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:26.285882 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df07c2-0056-4b05-bbea-e6e38c49932c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:26.294892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5244ad8-c7e8-477f-b409-621a51153ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:26.294892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5244ad8-c7e8-477f-b409-621a51153ebb admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:26.294892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5244ad8-c7e8-477f-b409-621a51153ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:26.294892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5244ad8-c7e8-477f-b409-621a51153ebb admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:26.295354 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5244ad8-c7e8-477f-b409-621a51153ebb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:26.295354 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5244ad8-c7e8-477f-b409-621a51153ebb admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:26.297133 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 801/1597] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:41:26.298032 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df07c2-0056-4b05-bbea-e6e38c49932c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:26.301269 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 797/1598] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:41:26.301968 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5244ad8-c7e8-477f-b409-621a51153ebb admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:26.308951 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df07c2-0056-4b05-bbea-e6e38c49932c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:26.311425 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5244ad8-c7e8-477f-b409-621a51153ebb admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:26.314440 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf1f05-0201-44a4-a3c2-2677276bac1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:26.314915 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf1f05-0201-44a4-a3c2-2677276bac1a admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:26.315188 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf1f05-0201-44a4-a3c2-2677276bac1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:26.315458 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf1f05-0201-44a4-a3c2-2677276bac1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:26.316093 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf1f05-0201-44a4-a3c2-2677276bac1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:26.316365 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f9f78-26b2-4f14-9524-bd5da418da08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:26.317071 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f9f78-26b2-4f14-9524-bd5da418da08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:26.317298 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f9f78-26b2-4f14-9524-bd5da418da08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:26.317563 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f9f78-26b2-4f14-9524-bd5da418da08 admin admin] PATH_INFO: `/v3/projects/2a4e5c5a950c4047b15749aace25f46d/users/00e07188d3df4d958ce83085b7bb141a/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:26.317774 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf1f05-0201-44a4-a3c2-2677276bac1a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:26.333632 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f9f78-26b2-4f14-9524-bd5da418da08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a4e5c5a950c4047b15749aace25f46d, user_id=00e07188d3df4d958ce83085b7bb141a, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:26.335359 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 802/1599] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:41:26.336111 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf1f05-0201-44a4-a3c2-2677276bac1a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:26.336752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f9f78-26b2-4f14-9524-bd5da418da08 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:26.348076 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf1f05-0201-44a4-a3c2-2677276bac1a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:26.353098 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9b718-0608-4100-a6af-e43a17737df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:26.355781 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9b718-0608-4100-a6af-e43a17737df5 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:26.355781 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9b718-0608-4100-a6af-e43a17737df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:26.355781 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9b718-0608-4100-a6af-e43a17737df5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:26.355781 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc9b718-0608-4100-a6af-e43a17737df5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:26.356846 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc9b718-0608-4100-a6af-e43a17737df5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:26.367857 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 803/1600] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:41:26.368601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9b718-0608-4100-a6af-e43a17737df5 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:26.379494 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9b718-0608-4100-a6af-e43a17737df5 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:26.384228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9562c7c-671b-4eb7-96a2-d5c17a311e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:26.386790 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9562c7c-671b-4eb7-96a2-d5c17a311e01 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:26.386790 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9562c7c-671b-4eb7-96a2-d5c17a311e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:26.386790 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9562c7c-671b-4eb7-96a2-d5c17a311e01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:26.386790 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9562c7c-671b-4eb7-96a2-d5c17a311e01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:26.387802 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9562c7c-671b-4eb7-96a2-d5c17a311e01 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:26.400010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 804/1601] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:41:26.400925 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9562c7c-671b-4eb7-96a2-d5c17a311e01 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:26.414257 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9562c7c-671b-4eb7-96a2-d5c17a311e01 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:26.421057 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 798/1602] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:41:26 2022] PUT /identity/v3/projects/2a4e5c5a950c4047b15749aace25f46d/users/00e07188d3df4d958ce83085b7bb141a/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:41:26.422438 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f9f78-26b2-4f14-9524-bd5da418da08 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:26.440155 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:26.481607 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08639981-0525-4481-89c3-5917e191d8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:26.482564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08639981-0525-4481-89c3-5917e191d8eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:26.482838 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08639981-0525-4481-89c3-5917e191d8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:26.483106 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08639981-0525-4481-89c3-5917e191d8eb admin admin] PATH_INFO: `/v3/projects/098b6977bfe046c89c058ad4263cf01a/users/4c14fdc6dfdc47ad91d595fabc996df2/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:26.490203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f9f78-26b2-4f14-9524-bd5da418da08 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:26.495895 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c512f3-5f64-43f6-9f06-91b69e9d3d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:26.495895 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c512f3-5f64-43f6-9f06-91b69e9d3d19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:26.495895 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c512f3-5f64-43f6-9f06-91b69e9d3d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:26.496185 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c512f3-5f64-43f6-9f06-91b69e9d3d19 admin admin] PATH_INFO: `/v3/projects/734d6a79fd5845f19569b2c2825fb38c/users/81d6a9d89e394e63ba5c8182a12b4142/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:26.501330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08639981-0525-4481-89c3-5917e191d8eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=098b6977bfe046c89c058ad4263cf01a, user_id=4c14fdc6dfdc47ad91d595fabc996df2, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:26.504129 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08639981-0525-4481-89c3-5917e191d8eb admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:26.513117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c512f3-5f64-43f6-9f06-91b69e9d3d19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=734d6a79fd5845f19569b2c2825fb38c, user_id=81d6a9d89e394e63ba5c8182a12b4142, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:26.520088 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c512f3-5f64-43f6-9f06-91b69e9d3d19 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:26.605842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 805/1603] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:41:26 2022] PUT /identity/v3/projects/098b6977bfe046c89c058ad4263cf01a/users/4c14fdc6dfdc47ad91d595fabc996df2/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:41:26.608021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08639981-0525-4481-89c3-5917e191d8eb admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:26.664652 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 799/1604] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:41:26 2022] PUT /identity/v3/projects/734d6a79fd5845f19569b2c2825fb38c/users/81d6a9d89e394e63ba5c8182a12b4142/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:41:26.664652 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c512f3-5f64-43f6-9f06-91b69e9d3d19 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:26.678866 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08639981-0525-4481-89c3-5917e191d8eb admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:26.726771 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:26.733029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c512f3-5f64-43f6-9f06-91b69e9d3d19 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:26.739142 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4ce2a-83ba-4285-aa32-84dc0c409cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:26.739761 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66396f-05bc-419e-af78-816c5e697276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:26.739888 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4ce2a-83ba-4285-aa32-84dc0c409cad admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:26.740054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4ce2a-83ba-4285-aa32-84dc0c409cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:26.740327 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4ce2a-83ba-4285-aa32-84dc0c409cad admin admin] PATH_INFO: `/v3/projects/0758667b3cf74cce904ee7f18a7784fa/users/05eb49c5c8834bbc866e2e636c76d7a4/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:26.740440 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66396f-05bc-419e-af78-816c5e697276 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:26.740703 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66396f-05bc-419e-af78-816c5e697276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:26.741002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66396f-05bc-419e-af78-816c5e697276 admin admin] PATH_INFO: `/v3/projects/da35d3f4baf549569c46e2eee8749e0b/users/6cbf838ebe8f4eb6a287448d30b55061/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:26.758952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4ce2a-83ba-4285-aa32-84dc0c409cad admin admin] RBAC: Authorizing `identity:create_grant(project_id=0758667b3cf74cce904ee7f18a7784fa, user_id=05eb49c5c8834bbc866e2e636c76d7a4, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:26.762045 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4ce2a-83ba-4285-aa32-84dc0c409cad admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:26.777172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66396f-05bc-419e-af78-816c5e697276 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da35d3f4baf549569c46e2eee8749e0b, user_id=6cbf838ebe8f4eb6a287448d30b55061, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:26.777172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66396f-05bc-419e-af78-816c5e697276 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:26.912229 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 800/1605] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:41:26 2022] PUT /identity/v3/projects/0758667b3cf74cce904ee7f18a7784fa/users/05eb49c5c8834bbc866e2e636c76d7a4/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:41:26.912229 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4ce2a-83ba-4285-aa32-84dc0c409cad admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:26.915017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 806/1606] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:41:26 2022] PUT /identity/v3/projects/da35d3f4baf549569c46e2eee8749e0b/users/6cbf838ebe8f4eb6a287448d30b55061/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:41:26.919226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e4ea4-3ee1-4d11-8c7b-6d76931d3bfc None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:26.919463 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e4ea4-3ee1-4d11-8c7b-6d76931d3bfc None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:26.920903 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e4ea4-3ee1-4d11-8c7b-6d76931d3bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:26.944943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-7c7e4ea4-3ee1-4d11-8c7b-6d76931d3bfc None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:26.946186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7c7e4ea4-3ee1-4d11-8c7b-6d76931d3bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:26.985612 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:26.997831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4ce2a-83ba-4285-aa32-84dc0c409cad admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:27.019802 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 807/1607] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:27.021047 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bacd69-9134-4995-9b1c-b2160badac0f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:27.021375 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bacd69-9134-4995-9b1c-b2160badac0f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:27.021655 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bacd69-9134-4995-9b1c-b2160badac0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:27.045508 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-30bacd69-9134-4995-9b1c-b2160badac0f None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:27.048070 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-30bacd69-9134-4995-9b1c-b2160badac0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:27.057120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 808/1608] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:27.058446 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a78597-89db-4195-9f44-570d3764ab3f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:27.058800 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a78597-89db-4195-9f44-570d3764ab3f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:27.059164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a78597-89db-4195-9f44-570d3764ab3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:27.080461 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce1314-e7ea-4548-9430-5ef0ba93e821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:27.081130 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce1314-e7ea-4548-9430-5ef0ba93e821 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:27.081446 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce1314-e7ea-4548-9430-5ef0ba93e821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:27.081728 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce1314-e7ea-4548-9430-5ef0ba93e821 admin admin] PATH_INFO: `/v3/projects/ce958d94ddc441e9bd10b5ae9638accc/users/32dcdc2e24e44a07ade44d1da330a77e/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:27.082531 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-05a78597-89db-4195-9f44-570d3764ab3f None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:27.084268 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-05a78597-89db-4195-9f44-570d3764ab3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:27.094477 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 809/1609] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:27.096036 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30021e-721f-4563-ada8-358c233a0798 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:27.096364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30021e-721f-4563-ada8-358c233a0798 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:27.096681 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30021e-721f-4563-ada8-358c233a0798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:27.099788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce1314-e7ea-4548-9430-5ef0ba93e821 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce958d94ddc441e9bd10b5ae9638accc, user_id=32dcdc2e24e44a07ade44d1da330a77e, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:27.102693 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce1314-e7ea-4548-9430-5ef0ba93e821 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:27.130317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-fb30021e-721f-4563-ada8-358c233a0798 None None] MFA Rules not processed for user `05eb49c5c8834bbc866e2e636c76d7a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:27.131725 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fb30021e-721f-4563-ada8-358c233a0798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:27.188434 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 801/1610] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:41:26 2022] PUT /identity/v3/projects/ce958d94ddc441e9bd10b5ae9638accc/users/32dcdc2e24e44a07ade44d1da330a77e/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:41:27.191626 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0000e-cb79-4de4-a26b-cc02d8b81a05 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:27.192391 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0000e-cb79-4de4-a26b-cc02d8b81a05 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:27.193103 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0000e-cb79-4de4-a26b-cc02d8b81a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:27.208958 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 810/1611] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:27.210034 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30021e-721f-4563-ada8-358c233a0798 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:27.231349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-3fb0000e-cb79-4de4-a26b-cc02d8b81a05 None None] MFA Rules not processed for user `6cbf838ebe8f4eb6a287448d30b55061`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:27.232846 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-3fb0000e-cb79-4de4-a26b-cc02d8b81a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:27.278197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30021e-721f-4563-ada8-358c233a0798 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:27.303690 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:27.318163 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 802/1612] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:27.318163 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb0000e-cb79-4de4-a26b-cc02d8b81a05 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:27.328412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cefe26b-371f-4b7d-8d45-91d2255900eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:27.329034 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cefe26b-371f-4b7d-8d45-91d2255900eb service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:27.329352 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cefe26b-371f-4b7d-8d45-91d2255900eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:27.329625 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cefe26b-371f-4b7d-8d45-91d2255900eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:27.351679 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:27.383245 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb0000e-cb79-4de4-a26b-cc02d8b81a05 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:27.387442 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98e994-7f8d-40dd-b36d-d0580678a393 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:27.387955 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98e994-7f8d-40dd-b36d-d0580678a393 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:27.388274 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98e994-7f8d-40dd-b36d-d0580678a393 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:27.388544 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98e994-7f8d-40dd-b36d-d0580678a393 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:27.398223 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cefe26b-371f-4b7d-8d45-91d2255900eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:27.399709 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c98e994-7f8d-40dd-b36d-d0580678a393 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:27.399992 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cefe26b-371f-4b7d-8d45-91d2255900eb service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:27.401333 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c98e994-7f8d-40dd-b36d-d0580678a393 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:27.411248 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 811/1613] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:41:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:41:27.415161 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cefe26b-371f-4b7d-8d45-91d2255900eb service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:27.420772 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 803/1614] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:41:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:41:27.428672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4998c-2285-497f-a8b3-8fb37a64b067 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:27.428672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4998c-2285-497f-a8b3-8fb37a64b067 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:27.428672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4998c-2285-497f-a8b3-8fb37a64b067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:27.430648 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cefe26b-371f-4b7d-8d45-91d2255900eb service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:27.435623 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a2b7f-920a-4537-877f-cd8f195bca03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:27.436458 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a2b7f-920a-4537-877f-cd8f195bca03 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:27.437690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a2b7f-920a-4537-877f-cd8f195bca03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:27.438069 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a2b7f-920a-4537-877f-cd8f195bca03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:27.449788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-05e4998c-2285-497f-a8b3-8fb37a64b067 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:27.450017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a2b7f-920a-4537-877f-cd8f195bca03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:27.451466 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-05e4998c-2285-497f-a8b3-8fb37a64b067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:27.452022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a2b7f-920a-4537-877f-cd8f195bca03 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:27.465088 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 812/1615] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:41:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:41:27.466756 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d73988-eaf3-4b1b-8b69-592d9f09ad89 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:27.467156 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d73988-eaf3-4b1b-8b69-592d9f09ad89 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:27.467520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d73988-eaf3-4b1b-8b69-592d9f09ad89 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:27.471066 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 804/1616] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:27.472059 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4998c-2285-497f-a8b3-8fb37a64b067 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:27.479415 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 813/1617] 10.222.0.33 () {54 vars in 962 bytes} [Mon Aug 8 08:41:27 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:41:27.481110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28229fba-3c54-4eea-85ca-1c164025e574 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:27.481557 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28229fba-3c54-4eea-85ca-1c164025e574 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:27.481931 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28229fba-3c54-4eea-85ca-1c164025e574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:27.485011 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4998c-2285-497f-a8b3-8fb37a64b067 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:27.491180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c15264-be00-40d1-9b6d-628847ea9554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:27.493256 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c15264-be00-40d1-9b6d-628847ea9554 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:27.493806 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c15264-be00-40d1-9b6d-628847ea9554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:27.494295 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c15264-be00-40d1-9b6d-628847ea9554 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:27.495242 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c15264-be00-40d1-9b6d-628847ea9554 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:27.497607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c15264-be00-40d1-9b6d-628847ea9554 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:27.527828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-28229fba-3c54-4eea-85ca-1c164025e574 None None] MFA Rules not processed for user `a3317f751ec74bde95ec262394001292`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:27.529120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-28229fba-3c54-4eea-85ca-1c164025e574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:27.606052 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 814/1618] 10.222.0.33 () {58 vars in 1059 bytes} [Mon Aug 8 08:41:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:27.606921 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28229fba-3c54-4eea-85ca-1c164025e574 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:27.615932 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28229fba-3c54-4eea-85ca-1c164025e574 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:27.619562 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95d73f-eb3e-46bd-b761-fe820c5625c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:27.620196 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95d73f-eb3e-46bd-b761-fe820c5625c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:27.620476 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95d73f-eb3e-46bd-b761-fe820c5625c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:27.620748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95d73f-eb3e-46bd-b761-fe820c5625c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:27.636713 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 805/1619] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:41:27 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:41:27.638552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c15264-be00-40d1-9b6d-628847ea9554 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:27.640914 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba95d73f-eb3e-46bd-b761-fe820c5625c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:27.643202 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba95d73f-eb3e-46bd-b761-fe820c5625c6 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:27.661212 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 815/1620] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:41:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:41:27.667182 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95d73f-eb3e-46bd-b761-fe820c5625c6 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:27.686177 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95d73f-eb3e-46bd-b761-fe820c5625c6 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:27.726241 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c15264-be00-40d1-9b6d-628847ea9554 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:27.754069 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c509d74-05c4-46bf-8444-1e1d03403256 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:27.754795 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c509d74-05c4-46bf-8444-1e1d03403256 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:27.757339 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c509d74-05c4-46bf-8444-1e1d03403256 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:27.757339 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c509d74-05c4-46bf-8444-1e1d03403256 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:27.770942 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee06ab0-9c0c-4b7c-a369-b4c750706387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:27.772854 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee06ab0-9c0c-4b7c-a369-b4c750706387 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:27.773444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee06ab0-9c0c-4b7c-a369-b4c750706387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:27.774055 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee06ab0-9c0c-4b7c-a369-b4c750706387 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:27.774946 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee06ab0-9c0c-4b7c-a369-b4c750706387 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:27.778177 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee06ab0-9c0c-4b7c-a369-b4c750706387 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:27.836142 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:27.844408 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 816/1621] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:41:27 2022] POST /identity/v3/users => generated 296 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:41:27.848547 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee06ab0-9c0c-4b7c-a369-b4c750706387 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:27.860999 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee06ab0-9c0c-4b7c-a369-b4c750706387 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:27.871327 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c509d74-05c4-46bf-8444-1e1d03403256 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:27.873340 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c509d74-05c4-46bf-8444-1e1d03403256 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:27.884922 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253553c4-9a8c-488a-aef4-d1a49cebca8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:27.890477 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253553c4-9a8c-488a-aef4-d1a49cebca8b admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:27.890477 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253553c4-9a8c-488a-aef4-d1a49cebca8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:27.890734 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 806/1622] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:41:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:41:27.890829 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253553c4-9a8c-488a-aef4-d1a49cebca8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:27.890829 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253553c4-9a8c-488a-aef4-d1a49cebca8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:27.893206 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253553c4-9a8c-488a-aef4-d1a49cebca8b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:27.921667 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c509d74-05c4-46bf-8444-1e1d03403256 service nova] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:27.932901 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 817/1623] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:27 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:41:27.936772 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c509d74-05c4-46bf-8444-1e1d03403256 service nova] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:27.943866 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f9b20e-6203-402e-9404-c97133aa6463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:27.944857 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9b20e-6203-402e-9404-c97133aa6463 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:27.945028 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9b20e-6203-402e-9404-c97133aa6463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:27.945438 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9b20e-6203-402e-9404-c97133aa6463 admin admin] PATH_INFO: `/v3/projects/d8a5e2a4a1fc4dd9a8c556c707a8dfe7/users/e616c5cbfe4946d78c242703c2f48429/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:27.962403 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f9b20e-6203-402e-9404-c97133aa6463 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8a5e2a4a1fc4dd9a8c556c707a8dfe7, user_id=e616c5cbfe4946d78c242703c2f48429, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:27.965976 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f9b20e-6203-402e-9404-c97133aa6463 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:27.995320 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:28.059643 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 807/1624] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:41:27 2022] PUT /identity/v3/projects/d8a5e2a4a1fc4dd9a8c556c707a8dfe7/users/e616c5cbfe4946d78c242703c2f48429/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:41:28.060191 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253553c4-9a8c-488a-aef4-d1a49cebca8b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:28.076411 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253553c4-9a8c-488a-aef4-d1a49cebca8b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:28.170851 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a81157-3e25-43f5-92fa-ce26d0dc0477 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:28.171475 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a81157-3e25-43f5-92fa-ce26d0dc0477 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:28.171901 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a81157-3e25-43f5-92fa-ce26d0dc0477 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:28.174231 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 808/1625] 10.222.0.33 () {54 vars in 912 bytes} [Mon Aug 8 08:41:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 08:41:28.178982 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a57bc-4a26-448f-82dc-2f526c2be568 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:28.179602 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a57bc-4a26-448f-82dc-2f526c2be568 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:28.180153 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a57bc-4a26-448f-82dc-2f526c2be568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:28.205122 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93899b43-007c-4f1c-9d2b-c7ddab662c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:28.205838 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93899b43-007c-4f1c-9d2b-c7ddab662c40 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:28.206962 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93899b43-007c-4f1c-9d2b-c7ddab662c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:28.206962 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93899b43-007c-4f1c-9d2b-c7ddab662c40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:28.207386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93899b43-007c-4f1c-9d2b-c7ddab662c40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:28.209270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93899b43-007c-4f1c-9d2b-c7ddab662c40 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:28.225832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-c11a57bc-4a26-448f-82dc-2f526c2be568 None None] MFA Rules not processed for user `a3317f751ec74bde95ec262394001292`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:28.230768 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 818/1626] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:28 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:41:28.231528 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c11a57bc-4a26-448f-82dc-2f526c2be568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:28.231945 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93899b43-007c-4f1c-9d2b-c7ddab662c40 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:28.253925 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93899b43-007c-4f1c-9d2b-c7ddab662c40 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:28.266802 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2017f1f-6718-4e22-9b74-99d7737bc43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:28.268083 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2017f1f-6718-4e22-9b74-99d7737bc43d admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:28.268607 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2017f1f-6718-4e22-9b74-99d7737bc43d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:28.269086 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2017f1f-6718-4e22-9b74-99d7737bc43d admin admin] PATH_INFO: `/v3/projects/d8a5e2a4a1fc4dd9a8c556c707a8dfe7/users/e616c5cbfe4946d78c242703c2f48429/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:28.272519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2017f1f-6718-4e22-9b74-99d7737bc43d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8a5e2a4a1fc4dd9a8c556c707a8dfe7, user_id=e616c5cbfe4946d78c242703c2f48429, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:28.277430 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2017f1f-6718-4e22-9b74-99d7737bc43d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:28.316341 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 819/1627] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:41:28 2022] PUT /identity/v3/projects/d8a5e2a4a1fc4dd9a8c556c707a8dfe7/users/e616c5cbfe4946d78c242703c2f48429/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:41:28.317990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237e6fb-4f8b-48cb-a620-469f9b74526d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:28.318228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237e6fb-4f8b-48cb-a620-469f9b74526d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:28.321744 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237e6fb-4f8b-48cb-a620-469f9b74526d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:28.387597 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d237e6fb-4f8b-48cb-a620-469f9b74526d None None] MFA Rules not processed for user `e616c5cbfe4946d78c242703c2f48429`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:28.391615 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d237e6fb-4f8b-48cb-a620-469f9b74526d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:28.398833 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 809/1628] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 08:41:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:28.419821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a57bc-4a26-448f-82dc-2f526c2be568 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:28.542188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a57bc-4a26-448f-82dc-2f526c2be568 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:28.587961 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 820/1629] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:28 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:28.600239 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237e6fb-4f8b-48cb-a620-469f9b74526d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:28.681244 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee654377-741c-4048-96b4-a03625a82dd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:28.684129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee654377-741c-4048-96b4-a03625a82dd6 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:28.684129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee654377-741c-4048-96b4-a03625a82dd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:28.684129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee654377-741c-4048-96b4-a03625a82dd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:28.752884 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237e6fb-4f8b-48cb-a620-469f9b74526d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:28.766263 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d21cc-a7ca-44c8-9148-2597dade783d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:28.767221 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d21cc-a7ca-44c8-9148-2597dade783d service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:28.767504 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d21cc-a7ca-44c8-9148-2597dade783d service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:28.767780 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d21cc-a7ca-44c8-9148-2597dade783d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:28.797508 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d21cc-a7ca-44c8-9148-2597dade783d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:28.798795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d21cc-a7ca-44c8-9148-2597dade783d service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:28.807091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:28.838567 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 821/1630] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:41:28 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:41:28.844799 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:28.854746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee654377-741c-4048-96b4-a03625a82dd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:28.857226 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee654377-741c-4048-96b4-a03625a82dd6 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:28.901928 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 810/1631] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:41:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:41:29.163415 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d21cc-a7ca-44c8-9148-2597dade783d service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:29.174392 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d21cc-a7ca-44c8-9148-2597dade783d service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:29.187049 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e17271-1ab9-44fd-a1e9-4f14046821df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:29.187848 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e17271-1ab9-44fd-a1e9-4f14046821df service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:29.188182 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e17271-1ab9-44fd-a1e9-4f14046821df service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:29.188459 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e17271-1ab9-44fd-a1e9-4f14046821df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:29.294453 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e17271-1ab9-44fd-a1e9-4f14046821df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:29.296532 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e17271-1ab9-44fd-a1e9-4f14046821df service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:29.332382 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 822/1632] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:41:29 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:41:29.856269 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dce07e-f3a7-42f4-922a-452c53913f3b None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:29.856855 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dce07e-f3a7-42f4-922a-452c53913f3b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:29.856855 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dce07e-f3a7-42f4-922a-452c53913f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:29.916034 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-b3dce07e-f3a7-42f4-922a-452c53913f3b None None] MFA Rules not processed for user `4c14fdc6dfdc47ad91d595fabc996df2`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:29.926554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b3dce07e-f3a7-42f4-922a-452c53913f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:30.118534 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 811/1633] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:29 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:30.119749 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e17271-1ab9-44fd-a1e9-4f14046821df service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:30.204580 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e17271-1ab9-44fd-a1e9-4f14046821df service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:30.210703 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ffd39-5836-4bda-9412-492f2a8ed528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:30.211627 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ffd39-5836-4bda-9412-492f2a8ed528 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:30.212092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ffd39-5836-4bda-9412-492f2a8ed528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:30.212683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ffd39-5836-4bda-9412-492f2a8ed528 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:30.213538 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ffd39-5836-4bda-9412-492f2a8ed528 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:30.215642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ffd39-5836-4bda-9412-492f2a8ed528 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:30.308175 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 823/1634] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:41:30 2022] POST /identity/v3/projects => generated 405 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:41:30.313338 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dce07e-f3a7-42f4-922a-452c53913f3b None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:30.329040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dce07e-f3a7-42f4-922a-452c53913f3b None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:30.406320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33287a6f-443d-4522-8238-7a98f74206df None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:30.406601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33287a6f-443d-4522-8238-7a98f74206df None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:30.418224 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33287a6f-443d-4522-8238-7a98f74206df None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:30.420345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 824/1635] 10.222.0.33 () {54 vars in 962 bytes} [Mon Aug 8 08:41:30 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:41:30.440201 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eececf-753f-42de-b806-b1c5ef83c684 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:30.441183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eececf-753f-42de-b806-b1c5ef83c684 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:30.441640 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eececf-753f-42de-b806-b1c5ef83c684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:30.516245 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-87eececf-753f-42de-b806-b1c5ef83c684 None None] MFA Rules not processed for user `a3317f751ec74bde95ec262394001292`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:30.521425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-87eececf-753f-42de-b806-b1c5ef83c684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:30.547713 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edda041-ee87-4971-9a5e-73d87f8b21a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:30.548517 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edda041-ee87-4971-9a5e-73d87f8b21a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:30.548796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edda041-ee87-4971-9a5e-73d87f8b21a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:30.549111 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edda041-ee87-4971-9a5e-73d87f8b21a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:30.549767 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edda041-ee87-4971-9a5e-73d87f8b21a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:30.564301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edda041-ee87-4971-9a5e-73d87f8b21a0 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:30.678253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 825/1636] 10.222.0.33 () {58 vars in 1059 bytes} [Mon Aug 8 08:41:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:30.683287 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d93170-a430-482a-b76a-06298c2ad154 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:30.684725 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d93170-a430-482a-b76a-06298c2ad154 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:30.685356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d93170-a430-482a-b76a-06298c2ad154 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:30.692343 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 826/1637] 10.222.0.33 () {54 vars in 912 bytes} [Mon Aug 8 08:41:30 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:41:30.697347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10416714-787f-4798-b9c5-69060dfdd445 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:30.697685 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10416714-787f-4798-b9c5-69060dfdd445 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:30.697896 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10416714-787f-4798-b9c5-69060dfdd445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:30.717434 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 812/1638] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:41:30 2022] POST /identity/v3/users => generated 309 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:41:30.725566 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edda041-ee87-4971-9a5e-73d87f8b21a0 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:30.735041 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-10416714-787f-4798-b9c5-69060dfdd445 None None] MFA Rules not processed for user `a3317f751ec74bde95ec262394001292`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:30.736526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-10416714-787f-4798-b9c5-69060dfdd445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:30.992493 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edda041-ee87-4971-9a5e-73d87f8b21a0 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:31.012427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 827/1639] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 08:41:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:31.014788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196fc4f-b4ac-4586-a241-1bc471c9e978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:31.014788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196fc4f-b4ac-4586-a241-1bc471c9e978 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:31.014788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196fc4f-b4ac-4586-a241-1bc471c9e978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:31.014788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196fc4f-b4ac-4586-a241-1bc471c9e978 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:31.018388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e196fc4f-b4ac-4586-a241-1bc471c9e978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:31.018388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e196fc4f-b4ac-4586-a241-1bc471c9e978 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:31.019562 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10416714-787f-4798-b9c5-69060dfdd445 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:31.021090 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 813/1640] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:30 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:41:31.021090 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196fc4f-b4ac-4586-a241-1bc471c9e978 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:31.022916 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:31.026301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196fc4f-b4ac-4586-a241-1bc471c9e978 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:31.026301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c18ba-31d3-4461-a53c-1b9d13c5e8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:31.026301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c18ba-31d3-4461-a53c-1b9d13c5e8bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:31.026301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c18ba-31d3-4461-a53c-1b9d13c5e8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:31.027391 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c18ba-31d3-4461-a53c-1b9d13c5e8bd admin admin] PATH_INFO: `/v3/projects/825433394de04f2eaba82207e42feb94/users/007f323efd7c499a95c64368d0b059d3/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:31.027391 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c18ba-31d3-4461-a53c-1b9d13c5e8bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=825433394de04f2eaba82207e42feb94, user_id=007f323efd7c499a95c64368d0b059d3, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:31.027836 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10416714-787f-4798-b9c5-69060dfdd445 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:31.028214 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c18ba-31d3-4461-a53c-1b9d13c5e8bd admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:31.028689 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467be2ab-2e84-440d-aa24-c647f0269a5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:31.028689 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467be2ab-2e84-440d-aa24-c647f0269a5d service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:31.028689 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467be2ab-2e84-440d-aa24-c647f0269a5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:31.028689 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467be2ab-2e84-440d-aa24-c647f0269a5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:31.029194 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467be2ab-2e84-440d-aa24-c647f0269a5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:31.029194 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467be2ab-2e84-440d-aa24-c647f0269a5d service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:31.029194 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 828/1641] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:41:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:41:31.126568 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 814/1642] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:41:30 2022] PUT /identity/v3/projects/825433394de04f2eaba82207e42feb94/users/007f323efd7c499a95c64368d0b059d3/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:41:31.127867 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467be2ab-2e84-440d-aa24-c647f0269a5d service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:31.146130 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467be2ab-2e84-440d-aa24-c647f0269a5d service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:31.261792 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf2d0a-5aa9-4833-93f2-7f51a2fa1e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:31.263019 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf2d0a-5aa9-4833-93f2-7f51a2fa1e85 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:31.263373 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf2d0a-5aa9-4833-93f2-7f51a2fa1e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:31.263804 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf2d0a-5aa9-4833-93f2-7f51a2fa1e85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:31.264662 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cf2d0a-5aa9-4833-93f2-7f51a2fa1e85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:31.266574 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cf2d0a-5aa9-4833-93f2-7f51a2fa1e85 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:31.278715 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 829/1643] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:41:31.283753 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c18ba-31d3-4461-a53c-1b9d13c5e8bd admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:31.295528 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c18ba-31d3-4461-a53c-1b9d13c5e8bd admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:31.304555 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fb8c79-d5b9-4155-9c10-3d6e0bce35ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:31.305442 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb8c79-d5b9-4155-9c10-3d6e0bce35ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:31.305721 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb8c79-d5b9-4155-9c10-3d6e0bce35ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:31.305994 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb8c79-d5b9-4155-9c10-3d6e0bce35ca admin admin] PATH_INFO: `/v3/projects/825433394de04f2eaba82207e42feb94/users/007f323efd7c499a95c64368d0b059d3/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:31.308766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fb8c79-d5b9-4155-9c10-3d6e0bce35ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=825433394de04f2eaba82207e42feb94, user_id=007f323efd7c499a95c64368d0b059d3, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:31.312958 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fb8c79-d5b9-4155-9c10-3d6e0bce35ca admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:31.331039 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 815/1644] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:41:31 2022] PUT /identity/v3/projects/825433394de04f2eaba82207e42feb94/users/007f323efd7c499a95c64368d0b059d3/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:41:34.922990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251aba1-0af7-48de-8f4c-26a49d654639 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:34.923500 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251aba1-0af7-48de-8f4c-26a49d654639 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:34.923566 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251aba1-0af7-48de-8f4c-26a49d654639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:35.000390 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e251aba1-0af7-48de-8f4c-26a49d654639 None None] MFA Rules not processed for user `007f323efd7c499a95c64368d0b059d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:35.001971 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e251aba1-0af7-48de-8f4c-26a49d654639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:35.168947 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 830/1645] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:34 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:38.173683 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef554e67-40ce-4422-9e8d-01d45a96e079 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:38.174609 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef554e67-40ce-4422-9e8d-01d45a96e079 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:38.175026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef554e67-40ce-4422-9e8d-01d45a96e079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:38.188562 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:38.251429 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-ef554e67-40ce-4422-9e8d-01d45a96e079 None None] MFA Rules not processed for user `b3e5a78c3b6c48d9a4aed7d36fd54736`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:38.252684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ef554e67-40ce-4422-9e8d-01d45a96e079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:38.386435 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 816/1646] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:38.399425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251aba1-0af7-48de-8f4c-26a49d654639 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:38.461323 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:38.548944 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251aba1-0af7-48de-8f4c-26a49d654639 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:38.567860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82818b58-e53d-40f7-9829-d8e1b60a3031 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:38.568787 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82818b58-e53d-40f7-9829-d8e1b60a3031 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:38.569099 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82818b58-e53d-40f7-9829-d8e1b60a3031 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:38.569797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82818b58-e53d-40f7-9829-d8e1b60a3031 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:38.580671 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82818b58-e53d-40f7-9829-d8e1b60a3031 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:38.583189 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82818b58-e53d-40f7-9829-d8e1b60a3031 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:38.604330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 831/1647] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:41:38 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:41:39.979671 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a50b5-4bcd-4d5e-9905-a2a6ea55aa08 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:39.980919 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a50b5-4bcd-4d5e-9905-a2a6ea55aa08 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:39.981379 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a50b5-4bcd-4d5e-9905-a2a6ea55aa08 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:39.987151 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 817/1648] 10.222.0.33 () {54 vars in 921 bytes} [Mon Aug 8 08:41:39 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:41:40.081928 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260d5d2-fd4d-465c-81d9-330954e22a2c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:40.082392 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260d5d2-fd4d-465c-81d9-330954e22a2c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:40.082392 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260d5d2-fd4d-465c-81d9-330954e22a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:40.150842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-3260d5d2-fd4d-465c-81d9-330954e22a2c None None] MFA Rules not processed for user `98c23a782b9b4144bace1248c3b2e754`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:40.162321 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3260d5d2-fd4d-465c-81d9-330954e22a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:40.327564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 832/1649] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Aug 8 08:41:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:40.531011 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520b706-34b0-483f-ba18-a24277a7592e None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:40.531642 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520b706-34b0-483f-ba18-a24277a7592e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:40.532099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520b706-34b0-483f-ba18-a24277a7592e None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:40.533919 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 818/1650] 10.222.0.33 () {54 vars in 961 bytes} [Mon Aug 8 08:41:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:41:40.539917 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3ddd2-43c1-4667-bb3f-7e8a3ece2fda None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:40.540402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3ddd2-43c1-4667-bb3f-7e8a3ece2fda None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:40.540879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3ddd2-43c1-4667-bb3f-7e8a3ece2fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:40.615147 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-37b3ddd2-43c1-4667-bb3f-7e8a3ece2fda None None] MFA Rules not processed for user `98c23a782b9b4144bace1248c3b2e754`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:40.618327 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-37b3ddd2-43c1-4667-bb3f-7e8a3ece2fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:40.709355 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 833/1651] 10.222.0.33 () {58 vars in 1058 bytes} [Mon Aug 8 08:41:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:41:40.723925 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520b706-34b0-483f-ba18-a24277a7592e None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:40.775649 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520b706-34b0-483f-ba18-a24277a7592e None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:40.801320 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004bfe29-2c9e-4315-80c0-3fb3654015da service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c23a782b9b4144bace1248c3b2e754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:40.807133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004bfe29-2c9e-4315-80c0-3fb3654015da service placement] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:40.807424 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004bfe29-2c9e-4315-80c0-3fb3654015da service placement] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:40.807696 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004bfe29-2c9e-4315-80c0-3fb3654015da service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:40.841735 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004bfe29-2c9e-4315-80c0-3fb3654015da service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:40.846929 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004bfe29-2c9e-4315-80c0-3fb3654015da service placement] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:40.884477 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 819/1652] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:41:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:41:41.885527 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1544a-4ab4-42be-8faf-e5cb6485b0cb None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:41.885990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1544a-4ab4-42be-8faf-e5cb6485b0cb None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:41.886074 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1544a-4ab4-42be-8faf-e5cb6485b0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:41.952504 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-82e1544a-4ab4-42be-8faf-e5cb6485b0cb None None] MFA Rules not processed for user `489aa74b7ab34d5789416739b8751473`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:41.954017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-82e1544a-4ab4-42be-8faf-e5cb6485b0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:42.150165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 834/1653] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:41:42.162502 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdb38e-535b-4bbc-9a8f-423a55334038 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:42.162961 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdb38e-535b-4bbc-9a8f-423a55334038 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:42.167280 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdb38e-535b-4bbc-9a8f-423a55334038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:42.240011 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-decdb38e-535b-4bbc-9a8f-423a55334038 None None] MFA Rules not processed for user `489aa74b7ab34d5789416739b8751473`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:42.254715 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-decdb38e-535b-4bbc-9a8f-423a55334038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:42.275664 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 820/1654] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:42.284981 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1544a-4ab4-42be-8faf-e5cb6485b0cb None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:42.306438 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1544a-4ab4-42be-8faf-e5cb6485b0cb None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:42.316522 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b1991-91de-41aa-bd4e-6cfac9f0f8f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:42.317265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b1991-91de-41aa-bd4e-6cfac9f0f8f5 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:42.317704 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b1991-91de-41aa-bd4e-6cfac9f0f8f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:42.318111 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b1991-91de-41aa-bd4e-6cfac9f0f8f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:42.332124 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b1991-91de-41aa-bd4e-6cfac9f0f8f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:42.332124 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b1991-91de-41aa-bd4e-6cfac9f0f8f5 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:42.336743 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decdb38e-535b-4bbc-9a8f-423a55334038 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:42.364268 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 835/1655] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:41:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:41:42.501723 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decdb38e-535b-4bbc-9a8f-423a55334038 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:42.516396 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47b449-3ab0-4346-9fa4-2d23813ce5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:42.517920 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47b449-3ab0-4346-9fa4-2d23813ce5e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:42.519445 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47b449-3ab0-4346-9fa4-2d23813ce5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:42.519905 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47b449-3ab0-4346-9fa4-2d23813ce5e7 admin admin] PATH_INFO: `/v3/users/05eb49c5c8834bbc866e2e636c76d7a4` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:42.523562 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47b449-3ab0-4346-9fa4-2d23813ce5e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05eb49c5c8834bbc866e2e636c76d7a4)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:42.528330 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47b449-3ab0-4346-9fa4-2d23813ce5e7 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:42.859595 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-df47b449-3ab0-4346-9fa4-2d23813ce5e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05eb49c5c8834bbc866e2e636c76d7a4'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:42.863786 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-df47b449-3ab0-4346-9fa4-2d23813ce5e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05eb49c5c8834bbc866e2e636c76d7a4'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:42.873932 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-df47b449-3ab0-4346-9fa4-2d23813ce5e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05eb49c5c8834bbc866e2e636c76d7a4'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:42.906146 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-df47b449-3ab0-4346-9fa4-2d23813ce5e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05eb49c5c8834bbc866e2e636c76d7a4'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:42.937083 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a35c6-d6f9-433d-abaa-9f1a46f19122 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:42.937776 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a35c6-d6f9-433d-abaa-9f1a46f19122 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:42.937776 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a35c6-d6f9-433d-abaa-9f1a46f19122 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:42.939180 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 836/1656] 10.222.0.33 () {54 vars in 921 bytes} [Mon Aug 8 08:41:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 08:41:42.959443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15117b-1ab1-4f9e-b607-f7cd5d5d6dcb None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:42.959996 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15117b-1ab1-4f9e-b607-f7cd5d5d6dcb None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:42.960447 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15117b-1ab1-4f9e-b607-f7cd5d5d6dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:43.008356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-7e15117b-1ab1-4f9e-b607-f7cd5d5d6dcb None None] MFA Rules not processed for user `98c23a782b9b4144bace1248c3b2e754`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:43.009114 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-df47b449-3ab0-4346-9fa4-2d23813ce5e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05eb49c5c8834bbc866e2e636c76d7a4'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:43.013779 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7e15117b-1ab1-4f9e-b607-f7cd5d5d6dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:43.029247 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:43.043507 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 837/1657] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Aug 8 08:41:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:43.049576 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 821/1658] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:41:42 2022] DELETE /identity/v3/users/05eb49c5c8834bbc866e2e636c76d7a4 => generated 0 bytes in 710 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:41:43.051167 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe92af-ad63-4eff-9d37-cd6693fb614a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:43.051617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe92af-ad63-4eff-9d37-cd6693fb614a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:43.052032 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe92af-ad63-4eff-9d37-cd6693fb614a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:43.100705 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-5efe92af-ad63-4eff-9d37-cd6693fb614a None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:43.102089 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5efe92af-ad63-4eff-9d37-cd6693fb614a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:43.119348 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:43.200664 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 838/1659] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:41:43.224058 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47b449-3ab0-4346-9fa4-2d23813ce5e7 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:43.226545 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-df47b449-3ab0-4346-9fa4-2d23813ce5e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:43.307144 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe92af-ad63-4eff-9d37-cd6693fb614a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:43.311160 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5efe92af-ad63-4eff-9d37-cd6693fb614a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:43.371289 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe92af-ad63-4eff-9d37-cd6693fb614a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:43.393710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fdd94-956d-4a56-b161-a53bead2dcc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:43.394375 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fdd94-956d-4a56-b161-a53bead2dcc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:43.394656 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fdd94-956d-4a56-b161-a53bead2dcc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:43.395809 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fdd94-956d-4a56-b161-a53bead2dcc4 admin admin] PATH_INFO: `/v3/users/4c14fdc6dfdc47ad91d595fabc996df2` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:43.402717 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0fdd94-956d-4a56-b161-a53bead2dcc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c14fdc6dfdc47ad91d595fabc996df2)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:43.403770 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: WARNING py.warnings [None req-7f0fdd94-956d-4a56-b161-a53bead2dcc4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 08:41:43.403770 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: warnings.warn(msg) Aug 08 08:41:43.403770 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: Aug 08 08:41:43.404991 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0fdd94-956d-4a56-b161-a53bead2dcc4 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:43.492374 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47b449-3ab0-4346-9fa4-2d23813ce5e7 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:43.512335 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041ebfe-c908-4ac3-a6ab-991ebb3b3abe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c23a782b9b4144bace1248c3b2e754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:43.513641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041ebfe-c908-4ac3-a6ab-991ebb3b3abe service placement] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:43.517104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041ebfe-c908-4ac3-a6ab-991ebb3b3abe service placement] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:43.517403 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041ebfe-c908-4ac3-a6ab-991ebb3b3abe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:43.539796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041ebfe-c908-4ac3-a6ab-991ebb3b3abe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:43.549416 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041ebfe-c908-4ac3-a6ab-991ebb3b3abe service placement] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:43.592999 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 822/1660] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:41:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:41:43.596761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041ebfe-c908-4ac3-a6ab-991ebb3b3abe service placement] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:43.599074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8041ebfe-c908-4ac3-a6ab-991ebb3b3abe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:43.735659 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7f0fdd94-956d-4a56-b161-a53bead2dcc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c14fdc6dfdc47ad91d595fabc996df2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:43.740155 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7f0fdd94-956d-4a56-b161-a53bead2dcc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c14fdc6dfdc47ad91d595fabc996df2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:43.743004 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7f0fdd94-956d-4a56-b161-a53bead2dcc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c14fdc6dfdc47ad91d595fabc996df2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:43.749421 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041ebfe-c908-4ac3-a6ab-991ebb3b3abe service placement] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:43.757748 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab68b1-5797-40bc-84f5-96a9b336be06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:43.758388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab68b1-5797-40bc-84f5-96a9b336be06 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:43.758668 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab68b1-5797-40bc-84f5-96a9b336be06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:43.758939 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab68b1-5797-40bc-84f5-96a9b336be06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:43.764331 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-62ab68b1-5797-40bc-84f5-96a9b336be06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:43.769348 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7f0fdd94-956d-4a56-b161-a53bead2dcc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c14fdc6dfdc47ad91d595fabc996df2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:43.794075 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab68b1-5797-40bc-84f5-96a9b336be06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:43.796891 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab68b1-5797-40bc-84f5-96a9b336be06 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:43.825953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 823/1661] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:41:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:41:43.832244 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bc2ff-f362-427c-862a-ee59f27b1ae2 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:43.832562 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bc2ff-f362-427c-862a-ee59f27b1ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:43.832846 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bc2ff-f362-427c-862a-ee59f27b1ae2 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:43.835187 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 824/1662] 10.222.0.33 () {54 vars in 912 bytes} [Mon Aug 8 08:41:43 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:41:43.842006 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03e53a-15e0-4e80-9819-201615bbf7c6 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:43.842291 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03e53a-15e0-4e80-9819-201615bbf7c6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:43.843766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03e53a-15e0-4e80-9819-201615bbf7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:43.859385 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7f0fdd94-956d-4a56-b161-a53bead2dcc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c14fdc6dfdc47ad91d595fabc996df2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:43.912347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 839/1663] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:41:43 2022] DELETE /identity/v3/users/4c14fdc6dfdc47ad91d595fabc996df2 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:41:43.929208 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-ae03e53a-15e0-4e80-9819-201615bbf7c6 None None] MFA Rules not processed for user `2c3b8b43026c40209bfc42acd2830610`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:43.930418 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ae03e53a-15e0-4e80-9819-201615bbf7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:44.079469 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 825/1664] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 08:41:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:44.091964 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fdd94-956d-4a56-b161-a53bead2dcc4 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:44.094082 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7f0fdd94-956d-4a56-b161-a53bead2dcc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:44.118904 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fdd94-956d-4a56-b161-a53bead2dcc4 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:44.123348 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae2781-77ad-4f80-af92-1ee9a6cd7aaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:44.124673 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae2781-77ad-4f80-af92-1ee9a6cd7aaa service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:44.129268 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae2781-77ad-4f80-af92-1ee9a6cd7aaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:44.129268 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae2781-77ad-4f80-af92-1ee9a6cd7aaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:44.164758 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae2781-77ad-4f80-af92-1ee9a6cd7aaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:44.166528 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae2781-77ad-4f80-af92-1ee9a6cd7aaa service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:44.190220 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 840/1665] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:41:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:41:45.144523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae03e53a-15e0-4e80-9819-201615bbf7c6 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:45.146056 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ae03e53a-15e0-4e80-9819-201615bbf7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:45.189920 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:45.303863 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae03e53a-15e0-4e80-9819-201615bbf7c6 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:45.317382 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ce5f5-c42c-4e54-aca6-c9e483a4bcc5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c23a782b9b4144bace1248c3b2e754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:45.319615 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ce5f5-c42c-4e54-aca6-c9e483a4bcc5 service placement] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:45.324871 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ce5f5-c42c-4e54-aca6-c9e483a4bcc5 service placement] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:45.325456 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ce5f5-c42c-4e54-aca6-c9e483a4bcc5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:45.329312 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f54ce5f5-c42c-4e54-aca6-c9e483a4bcc5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:45.378324 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ce5f5-c42c-4e54-aca6-c9e483a4bcc5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:45.378324 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ce5f5-c42c-4e54-aca6-c9e483a4bcc5 service placement] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:45.402158 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 826/1666] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:41:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:41:45.832838 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae2781-77ad-4f80-af92-1ee9a6cd7aaa service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:45.832838 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f3ae2781-77ad-4f80-af92-1ee9a6cd7aaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:45.947375 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ce5f5-c42c-4e54-aca6-c9e483a4bcc5 service placement] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:45.949230 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f54ce5f5-c42c-4e54-aca6-c9e483a4bcc5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:46.035672 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae2781-77ad-4f80-af92-1ee9a6cd7aaa service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:46.074424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42ee50-884d-4783-983e-6fa261b7a571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:46.079992 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42ee50-884d-4783-983e-6fa261b7a571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:46.080362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42ee50-884d-4783-983e-6fa261b7a571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:46.080645 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42ee50-884d-4783-983e-6fa261b7a571 admin admin] PATH_INFO: `/v3/projects/098b6977bfe046c89c058ad4263cf01a` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:46.085422 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a42ee50-884d-4783-983e-6fa261b7a571 admin admin] RBAC: Authorizing `identity:delete_project(project_id=098b6977bfe046c89c058ad4263cf01a)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:46.110757 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a42ee50-884d-4783-983e-6fa261b7a571 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:46.157734 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:46.181868 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ce5f5-c42c-4e54-aca6-c9e483a4bcc5 service placement] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:46.202095 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d3f28f-609e-42d9-8051-f27406caed25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:46.203189 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d3f28f-609e-42d9-8051-f27406caed25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:46.203834 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d3f28f-609e-42d9-8051-f27406caed25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:46.204404 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d3f28f-609e-42d9-8051-f27406caed25 admin admin] PATH_INFO: `/v3/users/6cbf838ebe8f4eb6a287448d30b55061` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:46.206033 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d3f28f-609e-42d9-8051-f27406caed25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cbf838ebe8f4eb6a287448d30b55061)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:46.211248 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d3f28f-609e-42d9-8051-f27406caed25 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:46.428300 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6a42ee50-884d-4783-983e-6fa261b7a571 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '098b6977bfe046c89c058ad4263cf01a', 'request_id': 'req-6a42ee50-884d-4783-983e-6fa261b7a571'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:46.469439 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a5d3f28f-609e-42d9-8051-f27406caed25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cbf838ebe8f4eb6a287448d30b55061'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:46.471670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a5d3f28f-609e-42d9-8051-f27406caed25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cbf838ebe8f4eb6a287448d30b55061'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:46.473207 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a5d3f28f-609e-42d9-8051-f27406caed25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cbf838ebe8f4eb6a287448d30b55061'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:46.508154 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a5d3f28f-609e-42d9-8051-f27406caed25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cbf838ebe8f4eb6a287448d30b55061'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:46.538255 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6a42ee50-884d-4783-983e-6fa261b7a571 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '098b6977bfe046c89c058ad4263cf01a', 'request_id': 'req-6a42ee50-884d-4783-983e-6fa261b7a571'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:46.545603 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6a42ee50-884d-4783-983e-6fa261b7a571 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '098b6977bfe046c89c058ad4263cf01a', 'request_id': 'req-6a42ee50-884d-4783-983e-6fa261b7a571'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:46.570800 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6a42ee50-884d-4783-983e-6fa261b7a571 admin admin] The token cache is being invalidate because project 098b6977bfe046c89c058ad4263cf01a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:41:46.577972 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6a42ee50-884d-4783-983e-6fa261b7a571 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:46.593509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6a42ee50-884d-4783-983e-6fa261b7a571 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:46.593509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 841/1667] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:41:45 2022] DELETE /identity/v3/projects/098b6977bfe046c89c058ad4263cf01a => generated 0 bytes in 761 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:41:46.593509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42ee50-884d-4783-983e-6fa261b7a571 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:46.595847 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6a42ee50-884d-4783-983e-6fa261b7a571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:46.648600 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a5d3f28f-609e-42d9-8051-f27406caed25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cbf838ebe8f4eb6a287448d30b55061'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:46.696464 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 827/1668] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:41:45 2022] DELETE /identity/v3/users/6cbf838ebe8f4eb6a287448d30b55061 => generated 0 bytes in 747 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:41:46.696464 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d3f28f-609e-42d9-8051-f27406caed25 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:46.701409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a5d3f28f-609e-42d9-8051-f27406caed25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:46.762786 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42ee50-884d-4783-983e-6fa261b7a571 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:46.784568 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:46.825132 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d3f28f-609e-42d9-8051-f27406caed25 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:46.847365 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c4b16a-ce97-49db-9fe3-b2e4af81c2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:46.848696 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4b16a-ce97-49db-9fe3-b2e4af81c2e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:46.849929 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4b16a-ce97-49db-9fe3-b2e4af81c2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:46.851274 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4b16a-ce97-49db-9fe3-b2e4af81c2e9 admin admin] PATH_INFO: `/v3/users/007f323efd7c499a95c64368d0b059d3` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:46.855680 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c4b16a-ce97-49db-9fe3-b2e4af81c2e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=007f323efd7c499a95c64368d0b059d3)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:46.862481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c4b16a-ce97-49db-9fe3-b2e4af81c2e9 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:46.909948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16533ad7-3985-40c8-928f-dad3b55fa793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:46.910799 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16533ad7-3985-40c8-928f-dad3b55fa793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:46.911192 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16533ad7-3985-40c8-928f-dad3b55fa793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:46.911601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16533ad7-3985-40c8-928f-dad3b55fa793 admin admin] PATH_INFO: `/v3/projects/0758667b3cf74cce904ee7f18a7784fa` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:46.914211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16533ad7-3985-40c8-928f-dad3b55fa793 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0758667b3cf74cce904ee7f18a7784fa)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:46.918959 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16533ad7-3985-40c8-928f-dad3b55fa793 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:47.161618 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-56c4b16a-ce97-49db-9fe3-b2e4af81c2e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '007f323efd7c499a95c64368d0b059d3'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:47.162555 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-56c4b16a-ce97-49db-9fe3-b2e4af81c2e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '007f323efd7c499a95c64368d0b059d3'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:47.163682 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-56c4b16a-ce97-49db-9fe3-b2e4af81c2e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '007f323efd7c499a95c64368d0b059d3'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:47.180868 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-16533ad7-3985-40c8-928f-dad3b55fa793 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0758667b3cf74cce904ee7f18a7784fa', 'request_id': 'req-16533ad7-3985-40c8-928f-dad3b55fa793'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:47.188013 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-56c4b16a-ce97-49db-9fe3-b2e4af81c2e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '007f323efd7c499a95c64368d0b059d3'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:47.196095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:47.326177 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-56c4b16a-ce97-49db-9fe3-b2e4af81c2e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '007f323efd7c499a95c64368d0b059d3'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:47.349341 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 828/1669] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:41:46 2022] DELETE /identity/v3/users/007f323efd7c499a95c64368d0b059d3 => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:41:47.350944 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453e466-3d53-487e-b150-78dcdc7af923 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:47.351355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453e466-3d53-487e-b150-78dcdc7af923 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:47.351907 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453e466-3d53-487e-b150-78dcdc7af923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:47.384628 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-16533ad7-3985-40c8-928f-dad3b55fa793 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0758667b3cf74cce904ee7f18a7784fa', 'request_id': 'req-16533ad7-3985-40c8-928f-dad3b55fa793'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:47.403899 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-a453e466-3d53-487e-b150-78dcdc7af923 None None] MFA Rules not processed for user `00e07188d3df4d958ce83085b7bb141a`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:47.405430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a453e466-3d53-487e-b150-78dcdc7af923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:47.413346 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-16533ad7-3985-40c8-928f-dad3b55fa793 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0758667b3cf74cce904ee7f18a7784fa', 'request_id': 'req-16533ad7-3985-40c8-928f-dad3b55fa793'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:47.439702 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-16533ad7-3985-40c8-928f-dad3b55fa793 admin admin] The token cache is being invalidate because project 0758667b3cf74cce904ee7f18a7784fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:41:47.440628 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-16533ad7-3985-40c8-928f-dad3b55fa793 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:47.441983 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-16533ad7-3985-40c8-928f-dad3b55fa793 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:47.448899 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 842/1670] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:41:46 2022] DELETE /identity/v3/projects/0758667b3cf74cce904ee7f18a7784fa => generated 0 bytes in 858 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:41:47.456894 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235fb61-d6b7-48fb-ba04-bfefdd08631d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:47.466929 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235fb61-d6b7-48fb-ba04-bfefdd08631d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:47.466929 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235fb61-d6b7-48fb-ba04-bfefdd08631d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:47.531045 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-f235fb61-d6b7-48fb-ba04-bfefdd08631d None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:47.536338 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f235fb61-d6b7-48fb-ba04-bfefdd08631d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:47.614388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 829/1671] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:47.621037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a764b-5527-417d-8fab-f59c184ffc6c None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:47.623346 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a764b-5527-417d-8fab-f59c184ffc6c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:47.624004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a764b-5527-417d-8fab-f59c184ffc6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:47.671285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-f43a764b-5527-417d-8fab-f59c184ffc6c None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:47.675874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f43a764b-5527-417d-8fab-f59c184ffc6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:47.677034 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 843/1672] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:47.680666 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7054143-668f-42f4-8aba-af4e72100ae5 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:47.680666 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7054143-668f-42f4-8aba-af4e72100ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:47.680666 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7054143-668f-42f4-8aba-af4e72100ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:47.699354 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 830/1673] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:47.700954 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a14420-9f2c-401b-b140-52af16cbcc56 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:47.700954 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a14420-9f2c-401b-b140-52af16cbcc56 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:47.702773 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a14420-9f2c-401b-b140-52af16cbcc56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:47.724387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-69a14420-9f2c-401b-b140-52af16cbcc56 None None] MFA Rules not processed for user `00e07188d3df4d958ce83085b7bb141a`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:47.726592 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-69a14420-9f2c-401b-b140-52af16cbcc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:47.739051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 831/1674] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:47.739051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a14420-9f2c-401b-b140-52af16cbcc56 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:47.739051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-69a14420-9f2c-401b-b140-52af16cbcc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:47.763554 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e7054143-668f-42f4-8aba-af4e72100ae5 None None] MFA Rules not processed for user `81d6a9d89e394e63ba5c8182a12b4142`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:47.765249 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e7054143-668f-42f4-8aba-af4e72100ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:47.794354 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:47.857799 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a14420-9f2c-401b-b140-52af16cbcc56 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:47.863315 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841003d-c0de-4437-aa38-c613063441d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:47.864281 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841003d-c0de-4437-aa38-c613063441d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:47.864756 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841003d-c0de-4437-aa38-c613063441d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:47.865072 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841003d-c0de-4437-aa38-c613063441d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:47.878906 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f841003d-c0de-4437-aa38-c613063441d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:47.880381 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f841003d-c0de-4437-aa38-c613063441d7 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:47.891172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 844/1675] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:47.892814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1c6f7-0f38-43f9-92ad-3109f9df6039 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:47.893388 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1c6f7-0f38-43f9-92ad-3109f9df6039 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:47.893860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1c6f7-0f38-43f9-92ad-3109f9df6039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:47.896641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 832/1676] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:41:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:41:47.897802 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841003d-c0de-4437-aa38-c613063441d7 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:47.899548 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f841003d-c0de-4437-aa38-c613063441d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:47.944627 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a4c1c6f7-0f38-43f9-92ad-3109f9df6039 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:47.945979 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a4c1c6f7-0f38-43f9-92ad-3109f9df6039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:47.957636 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 845/1677] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:47.962382 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1c6f7-0f38-43f9-92ad-3109f9df6039 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:47.964422 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a4c1c6f7-0f38-43f9-92ad-3109f9df6039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:48.113902 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841003d-c0de-4437-aa38-c613063441d7 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:48.126882 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506704b6-5115-4548-b7e8-d61570b59fa5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:48.127802 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506704b6-5115-4548-b7e8-d61570b59fa5 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:48.128247 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506704b6-5115-4548-b7e8-d61570b59fa5 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:48.129423 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506704b6-5115-4548-b7e8-d61570b59fa5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:48.140579 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506704b6-5115-4548-b7e8-d61570b59fa5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:48.142518 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506704b6-5115-4548-b7e8-d61570b59fa5 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:48.152122 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1c6f7-0f38-43f9-92ad-3109f9df6039 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:48.161310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 833/1678] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:41:47 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:41:48.162461 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506704b6-5115-4548-b7e8-d61570b59fa5 service nova] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:48.164058 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-506704b6-5115-4548-b7e8-d61570b59fa5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:48.176278 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df34111-27dd-4260-8fb8-166cb003d5c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:48.181194 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df34111-27dd-4260-8fb8-166cb003d5c1 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:48.181516 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df34111-27dd-4260-8fb8-166cb003d5c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:48.181805 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df34111-27dd-4260-8fb8-166cb003d5c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:48.183362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506704b6-5115-4548-b7e8-d61570b59fa5 service nova] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:48.191866 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e78c60-55c9-412e-8e13-18c779c1e00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:48.193672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e78c60-55c9-412e-8e13-18c779c1e00a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:48.193959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e78c60-55c9-412e-8e13-18c779c1e00a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:48.195881 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e78c60-55c9-412e-8e13-18c779c1e00a admin admin] PATH_INFO: `/v3/projects/825433394de04f2eaba82207e42feb94` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:48.199950 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e78c60-55c9-412e-8e13-18c779c1e00a admin admin] RBAC: Authorizing `identity:delete_project(project_id=825433394de04f2eaba82207e42feb94)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:48.200720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: WARNING py.warnings [None req-67e78c60-55c9-412e-8e13-18c779c1e00a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 08:41:48.200720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: warnings.warn(msg) Aug 08 08:41:48.200720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: Aug 08 08:41:48.204514 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e78c60-55c9-412e-8e13-18c779c1e00a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:48.207924 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df34111-27dd-4260-8fb8-166cb003d5c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:48.212636 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df34111-27dd-4260-8fb8-166cb003d5c1 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:48.239086 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 846/1679] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:41:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:41:48.252974 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df34111-27dd-4260-8fb8-166cb003d5c1 service nova] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:48.305433 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df34111-27dd-4260-8fb8-166cb003d5c1 service nova] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:48.319207 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:48.326320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba06d2-80dc-4f4e-91c0-5462b3f7c5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:48.327506 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba06d2-80dc-4f4e-91c0-5462b3f7c5bc admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:48.328338 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba06d2-80dc-4f4e-91c0-5462b3f7c5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:48.329469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba06d2-80dc-4f4e-91c0-5462b3f7c5bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:48.333922 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba06d2-80dc-4f4e-91c0-5462b3f7c5bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:41:48.334239 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba06d2-80dc-4f4e-91c0-5462b3f7c5bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:48.335888 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba06d2-80dc-4f4e-91c0-5462b3f7c5bc admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:48.383860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 847/1680] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:41:48.385304 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b49691-a25e-42e5-8066-e3450650db00 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:48.385594 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b49691-a25e-42e5-8066-e3450650db00 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:48.385866 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b49691-a25e-42e5-8066-e3450650db00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:48.404531 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-67e78c60-55c9-412e-8e13-18c779c1e00a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '825433394de04f2eaba82207e42feb94', 'request_id': 'req-67e78c60-55c9-412e-8e13-18c779c1e00a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:48.435337 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c0b49691-a25e-42e5-8066-e3450650db00 None None] MFA Rules not processed for user `b3e5a78c3b6c48d9a4aed7d36fd54736`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:48.437363 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c0b49691-a25e-42e5-8066-e3450650db00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:48.455694 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-67e78c60-55c9-412e-8e13-18c779c1e00a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '825433394de04f2eaba82207e42feb94', 'request_id': 'req-67e78c60-55c9-412e-8e13-18c779c1e00a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:48.479946 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-67e78c60-55c9-412e-8e13-18c779c1e00a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '825433394de04f2eaba82207e42feb94', 'request_id': 'req-67e78c60-55c9-412e-8e13-18c779c1e00a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:48.584892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-67e78c60-55c9-412e-8e13-18c779c1e00a admin admin] The token cache is being invalidate because project 825433394de04f2eaba82207e42feb94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:41:48.587189 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-67e78c60-55c9-412e-8e13-18c779c1e00a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:48.588260 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-67e78c60-55c9-412e-8e13-18c779c1e00a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:48.590473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 834/1681] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:41:48 2022] DELETE /identity/v3/projects/825433394de04f2eaba82207e42feb94 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:41:48.590608 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 848/1682] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:48.592311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e78c60-55c9-412e-8e13-18c779c1e00a admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:48.598012 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-67e78c60-55c9-412e-8e13-18c779c1e00a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:48.601577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfdf82-a69f-4866-9583-b449fb29df29 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:48.602765 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfdf82-a69f-4866-9583-b449fb29df29 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:48.603068 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfdf82-a69f-4866-9583-b449fb29df29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:48.650816 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c4dfdf82-a69f-4866-9583-b449fb29df29 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:48.656467 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c4dfdf82-a69f-4866-9583-b449fb29df29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:48.704278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:48.774977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e78c60-55c9-412e-8e13-18c779c1e00a admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:48.789338 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf560d8-e505-4fad-9dfa-2da3f78442e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:48.790073 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf560d8-e505-4fad-9dfa-2da3f78442e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:48.790425 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf560d8-e505-4fad-9dfa-2da3f78442e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:48.790763 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf560d8-e505-4fad-9dfa-2da3f78442e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:48.792016 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf560d8-e505-4fad-9dfa-2da3f78442e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:48.796692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf560d8-e505-4fad-9dfa-2da3f78442e2 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:48.850943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 849/1683] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:48.853264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c98d57-3794-49f0-8cd8-a28c62964b9d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:48.853559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c98d57-3794-49f0-8cd8-a28c62964b9d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:48.853840 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c98d57-3794-49f0-8cd8-a28c62964b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:48.897987 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-98c98d57-3794-49f0-8cd8-a28c62964b9d None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:48.900868 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-98c98d57-3794-49f0-8cd8-a28c62964b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:48.923397 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 850/1684] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:48.924589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c98d57-3794-49f0-8cd8-a28c62964b9d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:48.940686 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c98d57-3794-49f0-8cd8-a28c62964b9d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:48.956379 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 835/1685] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:41:48 2022] POST /identity/v3/projects => generated 405 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:41:48.956565 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da69d0-3c2c-4d62-8391-dc98aa594b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:48.956565 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da69d0-3c2c-4d62-8391-dc98aa594b4f admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:48.956565 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da69d0-3c2c-4d62-8391-dc98aa594b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:48.956565 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da69d0-3c2c-4d62-8391-dc98aa594b4f admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:48.957254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7da69d0-3c2c-4d62-8391-dc98aa594b4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:41:48.957254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7da69d0-3c2c-4d62-8391-dc98aa594b4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:48.957857 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf560d8-e505-4fad-9dfa-2da3f78442e2 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:48.958847 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7da69d0-3c2c-4d62-8391-dc98aa594b4f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:48.990375 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 851/1686] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:41:48.994718 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da69d0-3c2c-4d62-8391-dc98aa594b4f admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:49.024400 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da69d0-3c2c-4d62-8391-dc98aa594b4f admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:49.086537 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf560d8-e505-4fad-9dfa-2da3f78442e2 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:49.094007 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa8d11-5f75-4696-a6c6-7de164f64c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:49.094839 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa8d11-5f75-4696-a6c6-7de164f64c1c admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:49.095120 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa8d11-5f75-4696-a6c6-7de164f64c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:49.095435 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa8d11-5f75-4696-a6c6-7de164f64c1c admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:49.096529 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa8d11-5f75-4696-a6c6-7de164f64c1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:49.099493 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa8d11-5f75-4696-a6c6-7de164f64c1c admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:49.165088 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff4cd4-2a4e-48ca-aeef-c9c24f0d07bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:49.165819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff4cd4-2a4e-48ca-aeef-c9c24f0d07bd admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:49.166076 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff4cd4-2a4e-48ca-aeef-c9c24f0d07bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:49.166396 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff4cd4-2a4e-48ca-aeef-c9c24f0d07bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:49.167099 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff4cd4-2a4e-48ca-aeef-c9c24f0d07bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:49.169351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff4cd4-2a4e-48ca-aeef-c9c24f0d07bd admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:49.192940 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 836/1687] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:41:48 2022] POST /identity/v3/users => generated 303 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:41:49.192940 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4030ef-ec0a-4687-9741-0aa1d7142ee8 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:49.193487 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:49.193558 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4030ef-ec0a-4687-9741-0aa1d7142ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:49.193558 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4030ef-ec0a-4687-9741-0aa1d7142ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:49.268720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-8d4030ef-ec0a-4687-9741-0aa1d7142ee8 None None] MFA Rules not processed for user `32dcdc2e24e44a07ade44d1da330a77e`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:49.269964 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8d4030ef-ec0a-4687-9741-0aa1d7142ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:49.344971 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 852/1688] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:41:48 2022] POST /identity/v3/projects => generated 365 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:41:49.346909 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff4cd4-2a4e-48ca-aeef-c9c24f0d07bd admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:49.423306 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 837/1689] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:49.424592 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4030ef-ec0a-4687-9741-0aa1d7142ee8 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:49.427881 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8d4030ef-ec0a-4687-9741-0aa1d7142ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:49.449535 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4030ef-ec0a-4687-9741-0aa1d7142ee8 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:49.537724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:49.623427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff4cd4-2a4e-48ca-aeef-c9c24f0d07bd admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:49.624950 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f282b3f-ac10-482f-9555-f89b8fba9e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:49.625690 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f282b3f-ac10-482f-9555-f89b8fba9e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:49.625964 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f282b3f-ac10-482f-9555-f89b8fba9e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:49.626232 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f282b3f-ac10-482f-9555-f89b8fba9e2e admin admin] PATH_INFO: `/v3/projects/da35d3f4baf549569c46e2eee8749e0b` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:49.630200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f282b3f-ac10-482f-9555-f89b8fba9e2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=da35d3f4baf549569c46e2eee8749e0b)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:49.632212 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f282b3f-ac10-482f-9555-f89b8fba9e2e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:49.641322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a244e4-d7db-4ec0-99ad-0c3f82b36f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:49.641959 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a244e4-d7db-4ec0-99ad-0c3f82b36f73 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:49.642233 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a244e4-d7db-4ec0-99ad-0c3f82b36f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:49.643632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a244e4-d7db-4ec0-99ad-0c3f82b36f73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:49.644116 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a244e4-d7db-4ec0-99ad-0c3f82b36f73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:49.647858 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a244e4-d7db-4ec0-99ad-0c3f82b36f73 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:49.680619 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 853/1690] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:41:49.681593 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a244e4-d7db-4ec0-99ad-0c3f82b36f73 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:49.693053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a244e4-d7db-4ec0-99ad-0c3f82b36f73 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:49.699016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb67803-9dfb-4ff0-9d2f-b9700684ce06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:49.699993 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb67803-9dfb-4ff0-9d2f-b9700684ce06 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:49.699993 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb67803-9dfb-4ff0-9d2f-b9700684ce06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:49.700369 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb67803-9dfb-4ff0-9d2f-b9700684ce06 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:49.701662 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb67803-9dfb-4ff0-9d2f-b9700684ce06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:49.704750 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb67803-9dfb-4ff0-9d2f-b9700684ce06 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:49.853693 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 854/1691] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:41:49 2022] POST /identity/v3/users => generated 283 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:41:49.858872 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac363b-365b-480f-91f3-ccc16134cc9c None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:49.861513 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac363b-365b-480f-91f3-ccc16134cc9c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:49.872186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac363b-365b-480f-91f3-ccc16134cc9c None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:49.873708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 855/1692] 10.222.0.33 () {54 vars in 912 bytes} [Mon Aug 8 08:41:49 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:41:49.886447 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac363b-365b-480f-91f3-ccc16134cc9c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:49.913067 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-2f282b3f-ac10-482f-9555-f89b8fba9e2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da35d3f4baf549569c46e2eee8749e0b', 'request_id': 'req-2f282b3f-ac10-482f-9555-f89b8fba9e2e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:49.927167 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-2f282b3f-ac10-482f-9555-f89b8fba9e2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da35d3f4baf549569c46e2eee8749e0b', 'request_id': 'req-2f282b3f-ac10-482f-9555-f89b8fba9e2e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:49.951449 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-2f282b3f-ac10-482f-9555-f89b8fba9e2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da35d3f4baf549569c46e2eee8749e0b', 'request_id': 'req-2f282b3f-ac10-482f-9555-f89b8fba9e2e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:50.017640 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac363b-365b-480f-91f3-ccc16134cc9c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:50.022002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d10e9-16c5-499d-b507-3f4a3d8d748b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:50.022698 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d10e9-16c5-499d-b507-3f4a3d8d748b admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:50.023059 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d10e9-16c5-499d-b507-3f4a3d8d748b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:50.023404 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d10e9-16c5-499d-b507-3f4a3d8d748b admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:50.024755 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d10e9-16c5-499d-b507-3f4a3d8d748b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:50.026679 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d10e9-16c5-499d-b507-3f4a3d8d748b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:50.049112 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 856/1693] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:41:50.051561 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d10e9-16c5-499d-b507-3f4a3d8d748b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:50.066097 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d10e9-16c5-499d-b507-3f4a3d8d748b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:50.085997 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-2f282b3f-ac10-482f-9555-f89b8fba9e2e admin admin] The token cache is being invalidate because project da35d3f4baf549569c46e2eee8749e0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:41:50.085997 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-2f282b3f-ac10-482f-9555-f89b8fba9e2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:50.085997 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-2f282b3f-ac10-482f-9555-f89b8fba9e2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:50.089599 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5490e8-f3d8-412d-9660-0465cc0ea04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:50.089599 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5490e8-f3d8-412d-9660-0465cc0ea04f admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:50.094495 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 838/1694] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:41:49 2022] DELETE /identity/v3/projects/da35d3f4baf549569c46e2eee8749e0b => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:41:50.095285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5490e8-f3d8-412d-9660-0465cc0ea04f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:50.097039 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1960fb0-4a5b-4820-860a-3cc726eb8b5a None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:50.111518 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5490e8-f3d8-412d-9660-0465cc0ea04f admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:50.111518 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5490e8-f3d8-412d-9660-0465cc0ea04f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:50.111714 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1960fb0-4a5b-4820-860a-3cc726eb8b5a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:50.111904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1960fb0-4a5b-4820-860a-3cc726eb8b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:50.120293 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5490e8-f3d8-412d-9660-0465cc0ea04f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:50.184898 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 857/1695] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:41:50.190878 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5490e8-f3d8-412d-9660-0465cc0ea04f admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:50.209784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ba5490e8-f3d8-412d-9660-0465cc0ea04f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:50.222880 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-d1960fb0-4a5b-4820-860a-3cc726eb8b5a None None] MFA Rules not processed for user `2c3b8b43026c40209bfc42acd2830610`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:50.226200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d1960fb0-4a5b-4820-860a-3cc726eb8b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:50.252819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5490e8-f3d8-412d-9660-0465cc0ea04f admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:50.266653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9efa765-a942-4a52-a17c-28ddab7d042b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:50.270020 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9efa765-a942-4a52-a17c-28ddab7d042b admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:50.278471 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9efa765-a942-4a52-a17c-28ddab7d042b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:50.278751 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9efa765-a942-4a52-a17c-28ddab7d042b admin admin] PATH_INFO: `/v3/projects/dcc7eb78e2bb4c5a9a04bd3ee9207257/users/1b9eb1eb9516408f92e6f52f28cd22f9/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:50.332907 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9efa765-a942-4a52-a17c-28ddab7d042b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcc7eb78e2bb4c5a9a04bd3ee9207257, user_id=1b9eb1eb9516408f92e6f52f28cd22f9, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:50.336622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9efa765-a942-4a52-a17c-28ddab7d042b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:50.353974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 839/1696] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 08:41:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:50.355060 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1960fb0-4a5b-4820-860a-3cc726eb8b5a None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:50.357456 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d1960fb0-4a5b-4820-860a-3cc726eb8b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:50.383197 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1960fb0-4a5b-4820-860a-3cc726eb8b5a None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:50.391200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83f24d-3612-4f95-8864-198792b27078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:50.392213 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83f24d-3612-4f95-8864-198792b27078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:50.392676 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83f24d-3612-4f95-8864-198792b27078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:50.393174 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83f24d-3612-4f95-8864-198792b27078 admin admin] PATH_INFO: `/v3/users/e616c5cbfe4946d78c242703c2f48429` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:50.396205 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a83f24d-3612-4f95-8864-198792b27078 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e616c5cbfe4946d78c242703c2f48429)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:50.399251 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a83f24d-3612-4f95-8864-198792b27078 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:50.438214 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 858/1697] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:41:50 2022] PUT /identity/v3/projects/dcc7eb78e2bb4c5a9a04bd3ee9207257/users/1b9eb1eb9516408f92e6f52f28cd22f9/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:41:50.439259 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9efa765-a942-4a52-a17c-28ddab7d042b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:50.441301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b9efa765-a942-4a52-a17c-28ddab7d042b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:50.495860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:50.544064 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9efa765-a942-4a52-a17c-28ddab7d042b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:50.553234 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d5070c-d7d6-4208-aed3-2898552f70b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:50.553234 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5070c-d7d6-4208-aed3-2898552f70b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:50.553234 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5070c-d7d6-4208-aed3-2898552f70b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:50.553234 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5070c-d7d6-4208-aed3-2898552f70b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:50.553710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d5070c-d7d6-4208-aed3-2898552f70b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:50.555303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d5070c-d7d6-4208-aed3-2898552f70b1 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:50.576399 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 859/1698] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:41:50.581822 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d5070c-d7d6-4208-aed3-2898552f70b1 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:50.582281 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b8d5070c-d7d6-4208-aed3-2898552f70b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:50.688431 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d5070c-d7d6-4208-aed3-2898552f70b1 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:50.692297 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb3682-139b-42a5-a62a-40177f8731cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:50.693171 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb3682-139b-42a5-a62a-40177f8731cd service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:50.694742 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb3682-139b-42a5-a62a-40177f8731cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:50.694742 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb3682-139b-42a5-a62a-40177f8731cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:50.702146 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb3682-139b-42a5-a62a-40177f8731cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:50.704262 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb3682-139b-42a5-a62a-40177f8731cd service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:50.713271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 860/1699] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:41:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:41:50.715106 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e5220-e99c-475d-afd7-db563484b79b None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:50.715106 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e5220-e99c-475d-afd7-db563484b79b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:50.715277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e5220-e99c-475d-afd7-db563484b79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:50.757650 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4a83f24d-3612-4f95-8864-198792b27078 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e616c5cbfe4946d78c242703c2f48429'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:50.760051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4a83f24d-3612-4f95-8864-198792b27078 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e616c5cbfe4946d78c242703c2f48429'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:50.760734 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4a83f24d-3612-4f95-8864-198792b27078 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e616c5cbfe4946d78c242703c2f48429'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:50.771474 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-119e5220-e99c-475d-afd7-db563484b79b None None] MFA Rules not processed for user `1b9eb1eb9516408f92e6f52f28cd22f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:50.772826 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-119e5220-e99c-475d-afd7-db563484b79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:50.779110 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4a83f24d-3612-4f95-8864-198792b27078 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e616c5cbfe4946d78c242703c2f48429'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:50.858105 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4a83f24d-3612-4f95-8864-198792b27078 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e616c5cbfe4946d78c242703c2f48429'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:50.890378 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:50.896524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 861/1700] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:50 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:50.903408 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 840/1701] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:41:50 2022] DELETE /identity/v3/users/e616c5cbfe4946d78c242703c2f48429 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:41:50.907424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e5220-e99c-475d-afd7-db563484b79b None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:50.911027 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83f24d-3612-4f95-8864-198792b27078 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:50.912213 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-119e5220-e99c-475d-afd7-db563484b79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:50.914019 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-4a83f24d-3612-4f95-8864-198792b27078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:50.970034 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e5220-e99c-475d-afd7-db563484b79b None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:51.101235 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83f24d-3612-4f95-8864-198792b27078 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:51.115017 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4d8f2-9183-4232-88e8-5477ca48a411 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:51.115459 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4d8f2-9183-4232-88e8-5477ca48a411 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:51.115744 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4d8f2-9183-4232-88e8-5477ca48a411 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:51.116016 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4d8f2-9183-4232-88e8-5477ca48a411 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:51.118571 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-31a4d8f2-9183-4232-88e8-5477ca48a411 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:51.165068 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:51.172118 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a7b3b-784f-4d5b-b3eb-cd29f197ff50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:51.172956 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a7b3b-784f-4d5b-b3eb-cd29f197ff50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:51.173359 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a7b3b-784f-4d5b-b3eb-cd29f197ff50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:51.173744 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a7b3b-784f-4d5b-b3eb-cd29f197ff50 admin admin] PATH_INFO: `/v3/projects/6b06295b5f4e4e0abc0da9f00be58540/users/46aac537c3f34c0c8471f1ca6239f1a0/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:51.223472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a7b3b-784f-4d5b-b3eb-cd29f197ff50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b06295b5f4e4e0abc0da9f00be58540, user_id=46aac537c3f34c0c8471f1ca6239f1a0, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:51.223472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a7b3b-784f-4d5b-b3eb-cd29f197ff50 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:51.268056 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a4d8f2-9183-4232-88e8-5477ca48a411 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:51.269739 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a4d8f2-9183-4232-88e8-5477ca48a411 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:51.300285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 841/1702] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:41:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:41:51.302499 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23601930-c871-418d-bf1a-8b3733664b2c None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:51.303050 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23601930-c871-418d-bf1a-8b3733664b2c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:51.303542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23601930-c871-418d-bf1a-8b3733664b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:51.328284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 862/1703] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:41:50 2022] PUT /identity/v3/projects/6b06295b5f4e4e0abc0da9f00be58540/users/46aac537c3f34c0c8471f1ca6239f1a0/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:41:51.339692 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a7b3b-784f-4d5b-b3eb-cd29f197ff50 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:51.344843 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-057a7b3b-784f-4d5b-b3eb-cd29f197ff50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:51.441391 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-23601930-c871-418d-bf1a-8b3733664b2c None None] MFA Rules not processed for user `7f3422c6f9a041ceba851b71bd15d0df`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:51.443198 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-23601930-c871-418d-bf1a-8b3733664b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:51.518045 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a7b3b-784f-4d5b-b3eb-cd29f197ff50 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:51.540892 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae8332-8d06-4d1e-a62e-8f5e05b31230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:51.542345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae8332-8d06-4d1e-a62e-8f5e05b31230 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:51.542345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae8332-8d06-4d1e-a62e-8f5e05b31230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:51.542604 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae8332-8d06-4d1e-a62e-8f5e05b31230 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:51.545574 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae8332-8d06-4d1e-a62e-8f5e05b31230 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:51.546392 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae8332-8d06-4d1e-a62e-8f5e05b31230 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:51.638533 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 842/1704] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 08:41:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:51.649544 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf0239-3212-4699-bb43-fe6c25de27ed None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:51.650053 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf0239-3212-4699-bb43-fe6c25de27ed None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:51.650754 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf0239-3212-4699-bb43-fe6c25de27ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:51.681698 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:51.750320 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-caaf0239-3212-4699-bb43-fe6c25de27ed None None] MFA Rules not processed for user `46aac537c3f34c0c8471f1ca6239f1a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:51.751568 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-caaf0239-3212-4699-bb43-fe6c25de27ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:51.770536 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 863/1705] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:41:51 2022] POST /identity/v3/projects => generated 405 bytes in 439 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:41:51.772014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae8332-8d06-4d1e-a62e-8f5e05b31230 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:51.894722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 843/1706] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:51.899353 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf0239-3212-4699-bb43-fe6c25de27ed None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:51.916067 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf0239-3212-4699-bb43-fe6c25de27ed None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:51.974026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae8332-8d06-4d1e-a62e-8f5e05b31230 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:51.986616 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f131d4-7f94-4e16-9e55-9772c2bec526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:51.991039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f131d4-7f94-4e16-9e55-9772c2bec526 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:51.991412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f131d4-7f94-4e16-9e55-9772c2bec526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:51.992203 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f131d4-7f94-4e16-9e55-9772c2bec526 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:51.992944 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f131d4-7f94-4e16-9e55-9772c2bec526 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:51.998263 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f131d4-7f94-4e16-9e55-9772c2bec526 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:52.067942 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d4f5f-ad3f-436a-8c75-cb40a25c3282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:52.067942 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d4f5f-ad3f-436a-8c75-cb40a25c3282 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:52.068259 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d4f5f-ad3f-436a-8c75-cb40a25c3282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:52.068506 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d4f5f-ad3f-436a-8c75-cb40a25c3282 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:52.069174 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996d4f5f-ad3f-436a-8c75-cb40a25c3282 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:52.080132 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996d4f5f-ad3f-436a-8c75-cb40a25c3282 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:52.182473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 864/1707] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:41:51 2022] POST /identity/v3/users => generated 309 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:41:52.182473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128ed77-34c7-4e4a-b626-9e2b9d033df6 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:52.183012 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128ed77-34c7-4e4a-b626-9e2b9d033df6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:52.183012 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128ed77-34c7-4e4a-b626-9e2b9d033df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:52.237793 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 844/1708] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:41:51 2022] POST /identity/v3/projects => generated 363 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:41:52.238879 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d4f5f-ad3f-436a-8c75-cb40a25c3282 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:52.242027 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e128ed77-34c7-4e4a-b626-9e2b9d033df6 None None] MFA Rules not processed for user `00e07188d3df4d958ce83085b7bb141a`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:52.246571 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e128ed77-34c7-4e4a-b626-9e2b9d033df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:52.455455 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:52.465093 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d4f5f-ad3f-436a-8c75-cb40a25c3282 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:52.491636 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 865/1709] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:52.491953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54822ac-edfa-4ffd-879d-b24dacb4aaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:52.492521 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54822ac-edfa-4ffd-879d-b24dacb4aaa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:52.492626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e128ed77-34c7-4e4a-b626-9e2b9d033df6 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:52.492809 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54822ac-edfa-4ffd-879d-b24dacb4aaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:52.493088 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54822ac-edfa-4ffd-879d-b24dacb4aaa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:52.494831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54822ac-edfa-4ffd-879d-b24dacb4aaa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:52.498462 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54822ac-edfa-4ffd-879d-b24dacb4aaa1 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:52.500378 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e128ed77-34c7-4e4a-b626-9e2b9d033df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:52.538508 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 845/1710] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:41:52.541645 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54822ac-edfa-4ffd-879d-b24dacb4aaa1 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:52.585088 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54822ac-edfa-4ffd-879d-b24dacb4aaa1 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:52.612515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0fbda-5fbd-4302-a052-087a69477509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:52.613357 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0fbda-5fbd-4302-a052-087a69477509 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:52.613651 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0fbda-5fbd-4302-a052-087a69477509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:52.613926 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0fbda-5fbd-4302-a052-087a69477509 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:52.619561 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf0fbda-5fbd-4302-a052-087a69477509 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:52.619561 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf0fbda-5fbd-4302-a052-087a69477509 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:52.648585 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:52.763649 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e128ed77-34c7-4e4a-b626-9e2b9d033df6 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:52.770075 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 846/1711] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:41:52 2022] POST /identity/v3/users => generated 288 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:41:52.771117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0fbda-5fbd-4302-a052-087a69477509 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:52.799547 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a50f83-c624-4593-b1cd-d09a5c61dd6d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e69c0e02ad944f2a85fc5318018e5cb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:52.799547 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a50f83-c624-4593-b1cd-d09a5c61dd6d service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:52.799547 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a50f83-c624-4593-b1cd-d09a5c61dd6d service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:52.799547 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a50f83-c624-4593-b1cd-d09a5c61dd6d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:52.799932 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0fbda-5fbd-4302-a052-087a69477509 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:52.825064 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea3874e-4fd2-4afe-9d87-da481f44c573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:52.825790 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3874e-4fd2-4afe-9d87-da481f44c573 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:52.826270 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3874e-4fd2-4afe-9d87-da481f44c573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:52.827831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3874e-4fd2-4afe-9d87-da481f44c573 admin admin] PATH_INFO: `/v3/projects/00ac0ac7198d4c0f916d73d36aec840c/users/17332269b2e74c01a3a268a51424dab4/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:52.902958 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea3874e-4fd2-4afe-9d87-da481f44c573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00ac0ac7198d4c0f916d73d36aec840c, user_id=17332269b2e74c01a3a268a51424dab4, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:52.906392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea3874e-4fd2-4afe-9d87-da481f44c573 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:52.949646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a50f83-c624-4593-b1cd-d09a5c61dd6d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:52.951327 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a50f83-c624-4593-b1cd-d09a5c61dd6d service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:52.979349 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 866/1712] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 08:41:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 08:41:52.980611 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a50f83-c624-4593-b1cd-d09a5c61dd6d service swift] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:53.030546 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a50f83-c624-4593-b1cd-d09a5c61dd6d service swift] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:53.046158 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d054ebf-f273-4e7a-8c6a-24f6d639a3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:53.046158 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d054ebf-f273-4e7a-8c6a-24f6d639a3d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:53.046158 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d054ebf-f273-4e7a-8c6a-24f6d639a3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:53.046158 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d054ebf-f273-4e7a-8c6a-24f6d639a3d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:53.048792 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d054ebf-f273-4e7a-8c6a-24f6d639a3d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:53.050773 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d054ebf-f273-4e7a-8c6a-24f6d639a3d9 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:53.082023 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 867/1713] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:41:53.083943 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 847/1714] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:41:52 2022] PUT /identity/v3/projects/00ac0ac7198d4c0f916d73d36aec840c/users/17332269b2e74c01a3a268a51424dab4/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:41:53.088441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea3874e-4fd2-4afe-9d87-da481f44c573 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:53.090176 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d054ebf-f273-4e7a-8c6a-24f6d639a3d9 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:53.153535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d054ebf-f273-4e7a-8c6a-24f6d639a3d9 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:53.415682 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea3874e-4fd2-4afe-9d87-da481f44c573 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:53.429552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba110e92-744f-46f5-b8c0-baf7ba4b0d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:53.437071 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba110e92-744f-46f5-b8c0-baf7ba4b0d0c admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:53.437511 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba110e92-744f-46f5-b8c0-baf7ba4b0d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:53.437876 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba110e92-744f-46f5-b8c0-baf7ba4b0d0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:53.438607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba110e92-744f-46f5-b8c0-baf7ba4b0d0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:53.440265 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba110e92-744f-46f5-b8c0-baf7ba4b0d0c admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:53.494424 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 848/1715] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:53 2022] GET /identity/v3/roles => generated 1420 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:41:53.510115 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba110e92-744f-46f5-b8c0-baf7ba4b0d0c admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:53.522789 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce289b83-54fa-4814-ac58-6eb954e7c191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:53.523892 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce289b83-54fa-4814-ac58-6eb954e7c191 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:53.523892 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce289b83-54fa-4814-ac58-6eb954e7c191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:53.524084 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce289b83-54fa-4814-ac58-6eb954e7c191 admin admin] PATH_INFO: `/v3/projects/a7a1fbb085ae46348530bd3d6b764979/users/fed465dfa15444f8bc59db202a19f321/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:53.558088 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce289b83-54fa-4814-ac58-6eb954e7c191 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7a1fbb085ae46348530bd3d6b764979, user_id=fed465dfa15444f8bc59db202a19f321, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:53.561166 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce289b83-54fa-4814-ac58-6eb954e7c191 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:53.564623 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba110e92-744f-46f5-b8c0-baf7ba4b0d0c admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:53.568465 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:53.592263 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723205eb-1f2d-46ab-a161-03cd42e5ffda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:53.602688 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723205eb-1f2d-46ab-a161-03cd42e5ffda admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:53.604949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723205eb-1f2d-46ab-a161-03cd42e5ffda admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:53.605476 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723205eb-1f2d-46ab-a161-03cd42e5ffda admin admin] PATH_INFO: `/v3/projects/00ac0ac7198d4c0f916d73d36aec840c/users/17332269b2e74c01a3a268a51424dab4/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:53.616416 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723205eb-1f2d-46ab-a161-03cd42e5ffda admin admin] RBAC: Authorizing `identity:create_grant(project_id=00ac0ac7198d4c0f916d73d36aec840c, user_id=17332269b2e74c01a3a268a51424dab4, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:53.619370 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723205eb-1f2d-46ab-a161-03cd42e5ffda admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:53.671701 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 868/1716] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:41:53 2022] PUT /identity/v3/projects/a7a1fbb085ae46348530bd3d6b764979/users/fed465dfa15444f8bc59db202a19f321/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:41:53.675749 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce289b83-54fa-4814-ac58-6eb954e7c191 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:53.679669 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 849/1717] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:41:53 2022] PUT /identity/v3/projects/00ac0ac7198d4c0f916d73d36aec840c/users/17332269b2e74c01a3a268a51424dab4/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:41:53.691101 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32744b5-a7c7-4d46-8d3b-681747d6c220 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:53.692237 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32744b5-a7c7-4d46-8d3b-681747d6c220 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:53.692715 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32744b5-a7c7-4d46-8d3b-681747d6c220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:53.768311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-f32744b5-a7c7-4d46-8d3b-681747d6c220 None None] MFA Rules not processed for user `17332269b2e74c01a3a268a51424dab4`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:53.768311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f32744b5-a7c7-4d46-8d3b-681747d6c220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:53.850170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce289b83-54fa-4814-ac58-6eb954e7c191 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:53.861681 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95032b1-34a0-4898-ac49-894c5c6d833f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:53.864921 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95032b1-34a0-4898-ac49-894c5c6d833f admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:53.869117 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95032b1-34a0-4898-ac49-894c5c6d833f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:53.869474 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95032b1-34a0-4898-ac49-894c5c6d833f admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:53.871189 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95032b1-34a0-4898-ac49-894c5c6d833f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:53.872841 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95032b1-34a0-4898-ac49-894c5c6d833f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:53.976943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 869/1718] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:53 2022] GET /identity/v3/roles => generated 1420 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:41:53.978205 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95032b1-34a0-4898-ac49-894c5c6d833f admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:53.985695 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 850/1719] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:53 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:41:53.998016 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32744b5-a7c7-4d46-8d3b-681747d6c220 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:54.000311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f32744b5-a7c7-4d46-8d3b-681747d6c220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:54.028367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95032b1-34a0-4898-ac49-894c5c6d833f admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:54.049258 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a7138-e6fe-488b-b906-872064644a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:54.050363 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a7138-e6fe-488b-b906-872064644a53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:54.050363 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a7138-e6fe-488b-b906-872064644a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:54.050363 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a7138-e6fe-488b-b906-872064644a53 admin admin] PATH_INFO: `/v3/projects/a7a1fbb085ae46348530bd3d6b764979/users/fed465dfa15444f8bc59db202a19f321/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:54.055056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a7138-e6fe-488b-b906-872064644a53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7a1fbb085ae46348530bd3d6b764979, user_id=fed465dfa15444f8bc59db202a19f321, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:54.063595 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a7138-e6fe-488b-b906-872064644a53 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:54.091092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 870/1720] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:41:53 2022] PUT /identity/v3/projects/a7a1fbb085ae46348530bd3d6b764979/users/fed465dfa15444f8bc59db202a19f321/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:41:54.104892 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f43cbe-441d-401c-ac67-d741a7371806 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:54.114323 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f43cbe-441d-401c-ac67-d741a7371806 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:54.114323 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f43cbe-441d-401c-ac67-d741a7371806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:54.190356 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:54.244765 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32744b5-a7c7-4d46-8d3b-681747d6c220 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:54.245366 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-46f43cbe-441d-401c-ac67-d741a7371806 None None] MFA Rules not processed for user `fed465dfa15444f8bc59db202a19f321`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:54.273426 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-46f43cbe-441d-401c-ac67-d741a7371806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:54.429025 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ad480-dbe7-4f43-a300-2a4ffb808ec4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:54.429692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ad480-dbe7-4f43-a300-2a4ffb808ec4 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:54.429968 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ad480-dbe7-4f43-a300-2a4ffb808ec4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:54.430236 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ad480-dbe7-4f43-a300-2a4ffb808ec4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:54.485752 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 871/1721] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:54 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:54.488948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f43cbe-441d-401c-ac67-d741a7371806 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:54.492644 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-46f43cbe-441d-401c-ac67-d741a7371806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:54.519520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f43cbe-441d-401c-ac67-d741a7371806 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:54.532145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e66bc-beaa-404a-a293-895e4a8f6e93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:54.532825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e66bc-beaa-404a-a293-895e4a8f6e93 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:54.533046 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e66bc-beaa-404a-a293-895e4a8f6e93 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:54.533342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e66bc-beaa-404a-a293-895e4a8f6e93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:54.560201 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e66bc-beaa-404a-a293-895e4a8f6e93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:54.565498 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e66bc-beaa-404a-a293-895e4a8f6e93 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:54.617994 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 872/1722] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:41:54 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:41:54.648765 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ad480-dbe7-4f43-a300-2a4ffb808ec4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:54.650275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ad480-dbe7-4f43-a300-2a4ffb808ec4 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:54.699359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 851/1723] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:41:53 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 702 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:41:54.892668 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e66bc-beaa-404a-a293-895e4a8f6e93 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:54.896653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-400e66bc-beaa-404a-a293-895e4a8f6e93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:54.985849 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ad480-dbe7-4f43-a300-2a4ffb808ec4 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:54.997924 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ad480-dbe7-4f43-a300-2a4ffb808ec4 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:55.009751 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ae329-8700-405a-a12d-9a6bababeb5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:55.011662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ae329-8700-405a-a12d-9a6bababeb5e service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:55.015547 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ae329-8700-405a-a12d-9a6bababeb5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:55.015851 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ae329-8700-405a-a12d-9a6bababeb5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:55.385280 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:55.385280 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e66bc-beaa-404a-a293-895e4a8f6e93 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:55.385280 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c7d2f-343c-4d17-b5d3-a26f2633120e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:55.385280 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c7d2f-343c-4d17-b5d3-a26f2633120e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:55.386734 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c7d2f-343c-4d17-b5d3-a26f2633120e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:55.386734 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c7d2f-343c-4d17-b5d3-a26f2633120e admin admin] PATH_INFO: `/v3/projects/d8a5e2a4a1fc4dd9a8c556c707a8dfe7` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:55.386734 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c7d2f-343c-4d17-b5d3-a26f2633120e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8a5e2a4a1fc4dd9a8c556c707a8dfe7)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:55.386734 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c7d2f-343c-4d17-b5d3-a26f2633120e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:55.387739 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ae329-8700-405a-a12d-9a6bababeb5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:55.387739 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ae329-8700-405a-a12d-9a6bababeb5e service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:55.387739 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 852/1724] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:41:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:41:55.387739 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ae329-8700-405a-a12d-9a6bababeb5e service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:55.387739 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ae329-8700-405a-a12d-9a6bababeb5e service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:55.389057 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd438d-55c2-4806-b142-53678c68eb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:55.389057 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd438d-55c2-4806-b142-53678c68eb37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:55.389057 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd438d-55c2-4806-b142-53678c68eb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:55.389057 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd438d-55c2-4806-b142-53678c68eb37 admin admin] PATH_INFO: `/v3/users/1b9eb1eb9516408f92e6f52f28cd22f9` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:55.390128 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd438d-55c2-4806-b142-53678c68eb37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b9eb1eb9516408f92e6f52f28cd22f9)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:55.390128 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd438d-55c2-4806-b142-53678c68eb37 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:55.602585 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6d8c7d2f-343c-4d17-b5d3-a26f2633120e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8a5e2a4a1fc4dd9a8c556c707a8dfe7', 'request_id': 'req-6d8c7d2f-343c-4d17-b5d3-a26f2633120e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:55.723454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6d8c7d2f-343c-4d17-b5d3-a26f2633120e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8a5e2a4a1fc4dd9a8c556c707a8dfe7', 'request_id': 'req-6d8c7d2f-343c-4d17-b5d3-a26f2633120e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:55.736227 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6d8c7d2f-343c-4d17-b5d3-a26f2633120e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8a5e2a4a1fc4dd9a8c556c707a8dfe7', 'request_id': 'req-6d8c7d2f-343c-4d17-b5d3-a26f2633120e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:55.742558 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:55.767601 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a3dd438d-55c2-4806-b142-53678c68eb37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b9eb1eb9516408f92e6f52f28cd22f9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:55.773610 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a3dd438d-55c2-4806-b142-53678c68eb37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b9eb1eb9516408f92e6f52f28cd22f9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:55.775563 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6d8c7d2f-343c-4d17-b5d3-a26f2633120e admin admin] The token cache is being invalidate because project d8a5e2a4a1fc4dd9a8c556c707a8dfe7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:41:55.777521 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6d8c7d2f-343c-4d17-b5d3-a26f2633120e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:55.781100 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a3dd438d-55c2-4806-b142-53678c68eb37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b9eb1eb9516408f92e6f52f28cd22f9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:55.781710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6d8c7d2f-343c-4d17-b5d3-a26f2633120e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:55.789330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 873/1725] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:41:54 2022] DELETE /identity/v3/projects/d8a5e2a4a1fc4dd9a8c556c707a8dfe7 => generated 0 bytes in 899 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:41:55.808650 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a14f7d-5b11-4f76-83fe-d41d1a98d69f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:55.809064 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a14f7d-5b11-4f76-83fe-d41d1a98d69f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:55.809693 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a14f7d-5b11-4f76-83fe-d41d1a98d69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:55.814416 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a3dd438d-55c2-4806-b142-53678c68eb37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b9eb1eb9516408f92e6f52f28cd22f9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:55.893028 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a3dd438d-55c2-4806-b142-53678c68eb37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b9eb1eb9516408f92e6f52f28cd22f9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:41:55.935448 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-28a14f7d-5b11-4f76-83fe-d41d1a98d69f None None] MFA Rules not processed for user `81d6a9d89e394e63ba5c8182a12b4142`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:55.941170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-28a14f7d-5b11-4f76-83fe-d41d1a98d69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:55.962904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 853/1726] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:41:55 2022] DELETE /identity/v3/users/1b9eb1eb9516408f92e6f52f28cd22f9 => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:41:55.964334 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6a74b-fc83-4ea5-abcb-0301124a0c28 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:55.964649 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6a74b-fc83-4ea5-abcb-0301124a0c28 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:55.964908 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6a74b-fc83-4ea5-abcb-0301124a0c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:56.031652 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-18a6a74b-fc83-4ea5-abcb-0301124a0c28 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:56.033381 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-18a6a74b-fc83-4ea5-abcb-0301124a0c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:56.212457 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 874/1727] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 420 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:56.247251 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d2160-dbe1-4682-912d-f4b533a7d168 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:56.247690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d2160-dbe1-4682-912d-f4b533a7d168 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:56.249660 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d2160-dbe1-4682-912d-f4b533a7d168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:56.316763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8d1d2160-dbe1-4682-912d-f4b533a7d168 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:56.331673 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8d1d2160-dbe1-4682-912d-f4b533a7d168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:56.364239 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 854/1728] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:56.366214 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5dfa0-7b86-4d92-a732-89246e80d806 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:56.366664 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5dfa0-7b86-4d92-a732-89246e80d806 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:56.367735 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5dfa0-7b86-4d92-a732-89246e80d806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:56.454098 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-a1b5dfa0-7b86-4d92-a732-89246e80d806 None None] MFA Rules not processed for user `2c3b8b43026c40209bfc42acd2830610`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:56.454098 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a1b5dfa0-7b86-4d92-a732-89246e80d806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:56.551379 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 855/1729] 10.222.0.26 () {58 vars in 1015 bytes} [Mon Aug 8 08:41:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:56.554689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf573bd-d6ec-4ff1-92fb-bb7e38652b6e None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:56.555791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf573bd-d6ec-4ff1-92fb-bb7e38652b6e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:56.556344 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf573bd-d6ec-4ff1-92fb-bb7e38652b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:56.581417 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-aaf573bd-d6ec-4ff1-92fb-bb7e38652b6e None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:56.583071 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-aaf573bd-d6ec-4ff1-92fb-bb7e38652b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:56.602242 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 856/1730] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:56.609530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf573bd-d6ec-4ff1-92fb-bb7e38652b6e None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:56.622474 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf573bd-d6ec-4ff1-92fb-bb7e38652b6e None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:56.628669 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bef477-c2a1-4c36-b758-663fcabac738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:56.634126 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bef477-c2a1-4c36-b758-663fcabac738 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:56.634422 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bef477-c2a1-4c36-b758-663fcabac738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:56.634728 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bef477-c2a1-4c36-b758-663fcabac738 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:56.635412 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bef477-c2a1-4c36-b758-663fcabac738 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:41:56.635721 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bef477-c2a1-4c36-b758-663fcabac738 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:56.642237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 875/1731] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:56.643045 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bef477-c2a1-4c36-b758-663fcabac738 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:56.666641 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d2160-dbe1-4682-912d-f4b533a7d168 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:56.673188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 857/1732] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:41:56.677843 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8d1d2160-dbe1-4682-912d-f4b533a7d168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:56.678505 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bef477-c2a1-4c36-b758-663fcabac738 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:56.707060 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bef477-c2a1-4c36-b758-663fcabac738 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:56.727682 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81b401-0689-4a38-b9db-04cf707f5e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:56.728222 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81b401-0689-4a38-b9db-04cf707f5e3e admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:56.728506 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81b401-0689-4a38-b9db-04cf707f5e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:56.728779 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81b401-0689-4a38-b9db-04cf707f5e3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:56.729466 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81b401-0689-4a38-b9db-04cf707f5e3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:56.731572 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81b401-0689-4a38-b9db-04cf707f5e3e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:56.756902 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d2160-dbe1-4682-912d-f4b533a7d168 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:56.772130 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1efd85-44df-43ac-b124-aef35502eaaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:56.772785 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1efd85-44df-43ac-b124-aef35502eaaf service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:56.777316 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1efd85-44df-43ac-b124-aef35502eaaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:56.777633 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1efd85-44df-43ac-b124-aef35502eaaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:56.802635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1efd85-44df-43ac-b124-aef35502eaaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:56.820623 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1efd85-44df-43ac-b124-aef35502eaaf service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:56.868414 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 876/1733] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:41:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:41:56.869250 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1efd85-44df-43ac-b124-aef35502eaaf service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:56.886363 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 858/1734] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:41:56 2022] POST /identity/v3/projects => generated 387 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:41:56.892387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81b401-0689-4a38-b9db-04cf707f5e3e admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:56.907646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1efd85-44df-43ac-b124-aef35502eaaf service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:56.933304 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:56.972336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:57.040454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81b401-0689-4a38-b9db-04cf707f5e3e admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:57.078590 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6433cabf-68d8-4124-be4c-8686414f71c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:57.078590 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433cabf-68d8-4124-be4c-8686414f71c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:57.078590 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433cabf-68d8-4124-be4c-8686414f71c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:57.079044 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433cabf-68d8-4124-be4c-8686414f71c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:57.091007 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6433cabf-68d8-4124-be4c-8686414f71c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:57.092451 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6433cabf-68d8-4124-be4c-8686414f71c0 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:57.103056 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd577a-d254-40f4-9e4d-66909cfb6a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:57.106168 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd577a-d254-40f4-9e4d-66909cfb6a05 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:57.106168 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd577a-d254-40f4-9e4d-66909cfb6a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:57.106168 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd577a-d254-40f4-9e4d-66909cfb6a05 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:57.112204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 877/1735] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:41:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:41:57.115053 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd577a-d254-40f4-9e4d-66909cfb6a05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:57.133047 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd577a-d254-40f4-9e4d-66909cfb6a05 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:57.267089 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 859/1736] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:41:56 2022] POST /identity/v3/users => generated 294 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:41:57.271376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6433cabf-68d8-4124-be4c-8686414f71c0 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:57.298814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6433cabf-68d8-4124-be4c-8686414f71c0 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:57.346528 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f42ee-e372-4970-9a06-c171b0d31db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:57.347704 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f42ee-e372-4970-9a06-c171b0d31db7 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:57.348026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f42ee-e372-4970-9a06-c171b0d31db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:57.348330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f42ee-e372-4970-9a06-c171b0d31db7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:57.348964 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f42ee-e372-4970-9a06-c171b0d31db7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:57.351011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f42ee-e372-4970-9a06-c171b0d31db7 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:57.380941 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 878/1737] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:41:57.385462 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd577a-d254-40f4-9e4d-66909cfb6a05 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:57.400976 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd577a-d254-40f4-9e4d-66909cfb6a05 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:57.414263 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d390492-113d-43a5-bd2e-65030b244c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:57.416764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d390492-113d-43a5-bd2e-65030b244c49 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:57.416764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d390492-113d-43a5-bd2e-65030b244c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:57.416764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d390492-113d-43a5-bd2e-65030b244c49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:57.416764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d390492-113d-43a5-bd2e-65030b244c49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:57.417844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d390492-113d-43a5-bd2e-65030b244c49 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:57.456949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 860/1738] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:41:57.461284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f42ee-e372-4970-9a06-c171b0d31db7 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:57.477516 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f42ee-e372-4970-9a06-c171b0d31db7 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:57.503496 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03990c50-5165-4c6b-aa74-a13a6f5e37ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:57.503655 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03990c50-5165-4c6b-aa74-a13a6f5e37ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:57.504481 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03990c50-5165-4c6b-aa74-a13a6f5e37ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:57.504766 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03990c50-5165-4c6b-aa74-a13a6f5e37ea admin admin] PATH_INFO: `/v3/projects/bf9a5deb6e6a4eb0bbc392f8873348d7/users/64de9542e22c49a78cad9d9ce12e3dce/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:57.555786 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03990c50-5165-4c6b-aa74-a13a6f5e37ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf9a5deb6e6a4eb0bbc392f8873348d7, user_id=64de9542e22c49a78cad9d9ce12e3dce, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:57.565167 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03990c50-5165-4c6b-aa74-a13a6f5e37ea admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:57.704921 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 879/1739] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:41:57 2022] PUT /identity/v3/projects/bf9a5deb6e6a4eb0bbc392f8873348d7/users/64de9542e22c49a78cad9d9ce12e3dce/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:41:57.712318 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28fd86-c1cb-4797-830b-0d5c10df1d54 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:57.712849 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28fd86-c1cb-4797-830b-0d5c10df1d54 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:57.713436 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28fd86-c1cb-4797-830b-0d5c10df1d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:57.812501 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-5b28fd86-c1cb-4797-830b-0d5c10df1d54 None None] MFA Rules not processed for user `64de9542e22c49a78cad9d9ce12e3dce`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:57.814148 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5b28fd86-c1cb-4797-830b-0d5c10df1d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:58.046971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 861/1740] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:58.051949 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03990c50-5165-4c6b-aa74-a13a6f5e37ea admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:58.241834 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03990c50-5165-4c6b-aa74-a13a6f5e37ea admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:58.259434 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4e2d2d-4032-41c7-a2f2-0bd6a1a8a62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:58.260400 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e2d2d-4032-41c7-a2f2-0bd6a1a8a62d admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:58.261490 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e2d2d-4032-41c7-a2f2-0bd6a1a8a62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:58.267773 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e2d2d-4032-41c7-a2f2-0bd6a1a8a62d admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:58.267773 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4e2d2d-4032-41c7-a2f2-0bd6a1a8a62d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:58.268880 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4e2d2d-4032-41c7-a2f2-0bd6a1a8a62d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:58.454223 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 880/1741] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:41:58 2022] POST /identity/v3/projects => generated 387 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:41:58.454750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28fd86-c1cb-4797-830b-0d5c10df1d54 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:58.476385 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28fd86-c1cb-4797-830b-0d5c10df1d54 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:58.619250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a9bc3-04a3-4bf0-9cbb-1dec3673388e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:58.621099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a9bc3-04a3-4bf0-9cbb-1dec3673388e admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:58.628484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a9bc3-04a3-4bf0-9cbb-1dec3673388e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:58.629052 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a9bc3-04a3-4bf0-9cbb-1dec3673388e admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:58.629984 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a9bc3-04a3-4bf0-9cbb-1dec3673388e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:58.633263 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a9bc3-04a3-4bf0-9cbb-1dec3673388e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:58.741333 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 862/1742] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:41:58 2022] POST /identity/v3/users => generated 300 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:41:58.751683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4e2d2d-4032-41c7-a2f2-0bd6a1a8a62d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:58.774672 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4e2d2d-4032-41c7-a2f2-0bd6a1a8a62d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:58.783230 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e44e6b2-462b-46eb-8071-ae293f01c397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:58.783881 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44e6b2-462b-46eb-8071-ae293f01c397 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:58.784426 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44e6b2-462b-46eb-8071-ae293f01c397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:58.784717 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44e6b2-462b-46eb-8071-ae293f01c397 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:58.785581 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e44e6b2-462b-46eb-8071-ae293f01c397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:58.787330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e44e6b2-462b-46eb-8071-ae293f01c397 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:58.808940 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 881/1743] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:41:58.813365 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a9bc3-04a3-4bf0-9cbb-1dec3673388e admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:58.838105 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a9bc3-04a3-4bf0-9cbb-1dec3673388e admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:58.866647 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abac92e-d9ec-4646-b656-f86a27198807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:58.867164 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abac92e-d9ec-4646-b656-f86a27198807 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:58.867450 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abac92e-d9ec-4646-b656-f86a27198807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:58.867723 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abac92e-d9ec-4646-b656-f86a27198807 admin admin] PATH_INFO: `/v3/projects/3e222d375e7d4baebec0b763db8c2113/users/8a4635cd1e384ff397b6d5d29ef325f8/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:58.900974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abac92e-d9ec-4646-b656-f86a27198807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e222d375e7d4baebec0b763db8c2113, user_id=8a4635cd1e384ff397b6d5d29ef325f8, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:58.914351 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abac92e-d9ec-4646-b656-f86a27198807 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:59.013610 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:59.021172 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 863/1744] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:41:58 2022] PUT /identity/v3/projects/3e222d375e7d4baebec0b763db8c2113/users/8a4635cd1e384ff397b6d5d29ef325f8/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:41:59.024922 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e44e6b2-462b-46eb-8071-ae293f01c397 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:59.050237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e44e6b2-462b-46eb-8071-ae293f01c397 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:59.269285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:41:59.304545 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2241146-3c94-43ec-baca-a452260c7db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:59.315910 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2241146-3c94-43ec-baca-a452260c7db7 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:59.316788 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2241146-3c94-43ec-baca-a452260c7db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:59.317881 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2241146-3c94-43ec-baca-a452260c7db7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:59.321289 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2241146-3c94-43ec-baca-a452260c7db7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:59.324572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2241146-3c94-43ec-baca-a452260c7db7 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:59.368777 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 882/1745] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:41:59.375531 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abac92e-d9ec-4646-b656-f86a27198807 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:59.394294 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abac92e-d9ec-4646-b656-f86a27198807 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:59.403146 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a727e-2a82-48c2-8c38-adf240908091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:59.403913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a727e-2a82-48c2-8c38-adf240908091 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:59.404195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a727e-2a82-48c2-8c38-adf240908091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:59.404472 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a727e-2a82-48c2-8c38-adf240908091 admin admin] PATH_INFO: `/v3/projects/3e222d375e7d4baebec0b763db8c2113/users/8a4635cd1e384ff397b6d5d29ef325f8/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:59.407232 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242a727e-2a82-48c2-8c38-adf240908091 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e222d375e7d4baebec0b763db8c2113, user_id=8a4635cd1e384ff397b6d5d29ef325f8, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:59.423671 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242a727e-2a82-48c2-8c38-adf240908091 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:59.470329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 864/1746] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:41:59 2022] PUT /identity/v3/projects/3e222d375e7d4baebec0b763db8c2113/users/8a4635cd1e384ff397b6d5d29ef325f8/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:41:59.482536 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63c044-9da3-4278-a480-c35625426ded None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:59.482827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63c044-9da3-4278-a480-c35625426ded None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:59.483858 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63c044-9da3-4278-a480-c35625426ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:59.529640 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-3a63c044-9da3-4278-a480-c35625426ded None None] MFA Rules not processed for user `8a4635cd1e384ff397b6d5d29ef325f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:41:59.531771 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3a63c044-9da3-4278-a480-c35625426ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:59.664293 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 883/1747] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:41:59 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:41:59.675784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a727e-2a82-48c2-8c38-adf240908091 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:41:59.678575 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-242a727e-2a82-48c2-8c38-adf240908091 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:41:59.789022 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a727e-2a82-48c2-8c38-adf240908091 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:41:59.795632 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b6d3b-1fe6-4149-bc85-a40d194a3628 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:41:59.798713 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b6d3b-1fe6-4149-bc85-a40d194a3628 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:41:59.799395 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b6d3b-1fe6-4149-bc85-a40d194a3628 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:41:59.804682 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b6d3b-1fe6-4149-bc85-a40d194a3628 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:41:59.832125 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b6d3b-1fe6-4149-bc85-a40d194a3628 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:41:59.833751 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b6d3b-1fe6-4149-bc85-a40d194a3628 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:41:59.879556 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 865/1748] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:41:59 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:41:59.938325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63c044-9da3-4278-a480-c35625426ded None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:00.065796 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63c044-9da3-4278-a480-c35625426ded None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:00.072385 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730724b3-7670-4c21-b38e-280ee080aa7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:00.073048 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730724b3-7670-4c21-b38e-280ee080aa7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:00.073373 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730724b3-7670-4c21-b38e-280ee080aa7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:00.073648 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730724b3-7670-4c21-b38e-280ee080aa7c admin admin] PATH_INFO: `/v3/users/64de9542e22c49a78cad9d9ce12e3dce` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:00.074877 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730724b3-7670-4c21-b38e-280ee080aa7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=64de9542e22c49a78cad9d9ce12e3dce)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:00.077344 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730724b3-7670-4c21-b38e-280ee080aa7c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:00.370000 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-730724b3-7670-4c21-b38e-280ee080aa7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64de9542e22c49a78cad9d9ce12e3dce'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:00.370675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-730724b3-7670-4c21-b38e-280ee080aa7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64de9542e22c49a78cad9d9ce12e3dce'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:00.376610 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-730724b3-7670-4c21-b38e-280ee080aa7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64de9542e22c49a78cad9d9ce12e3dce'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:00.419548 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-730724b3-7670-4c21-b38e-280ee080aa7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64de9542e22c49a78cad9d9ce12e3dce'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:00.516834 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-730724b3-7670-4c21-b38e-280ee080aa7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64de9542e22c49a78cad9d9ce12e3dce'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:00.556506 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:00.563874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 884/1749] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:41:59 2022] DELETE /identity/v3/users/64de9542e22c49a78cad9d9ce12e3dce => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:00.568507 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22fe619-2856-48d9-bf9f-fb8cf95617de None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:00.569508 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22fe619-2856-48d9-bf9f-fb8cf95617de None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:00.569938 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22fe619-2856-48d9-bf9f-fb8cf95617de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:00.615494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-e22fe619-2856-48d9-bf9f-fb8cf95617de None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:00.626696 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e22fe619-2856-48d9-bf9f-fb8cf95617de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:00.806555 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 866/1750] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:00.830559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730724b3-7670-4c21-b38e-280ee080aa7c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:00.836111 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-730724b3-7670-4c21-b38e-280ee080aa7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:00.958826 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730724b3-7670-4c21-b38e-280ee080aa7c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:00.966731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3972c0ab-f6d2-4d8c-83ab-e285f3879d71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:00.968130 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972c0ab-f6d2-4d8c-83ab-e285f3879d71 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:00.974256 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972c0ab-f6d2-4d8c-83ab-e285f3879d71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:00.974689 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972c0ab-f6d2-4d8c-83ab-e285f3879d71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:00.998194 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3972c0ab-f6d2-4d8c-83ab-e285f3879d71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:01.000737 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3972c0ab-f6d2-4d8c-83ab-e285f3879d71 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:01.029341 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 885/1751] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:42:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:42:01.631133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415e06e-9dbe-4638-a164-42ef8da2cdc4 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:01.633480 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415e06e-9dbe-4638-a164-42ef8da2cdc4 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:01.633480 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415e06e-9dbe-4638-a164-42ef8da2cdc4 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:01.635244 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 867/1752] 10.222.0.33 () {54 vars in 921 bytes} [Mon Aug 8 08:42:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:42:01.673077 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96bd0d-e3d8-4832-aea6-761f916b66e8 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:01.673656 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96bd0d-e3d8-4832-aea6-761f916b66e8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:01.674102 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96bd0d-e3d8-4832-aea6-761f916b66e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:01.700863 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415e06e-9dbe-4638-a164-42ef8da2cdc4 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:01.703239 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a415e06e-9dbe-4638-a164-42ef8da2cdc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:01.737649 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ad96bd0d-e3d8-4832-aea6-761f916b66e8 None None] MFA Rules not processed for user `a3317f751ec74bde95ec262394001292`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:01.740942 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ad96bd0d-e3d8-4832-aea6-761f916b66e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:01.758601 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415e06e-9dbe-4638-a164-42ef8da2cdc4 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:01.783850 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befd37b-4641-4fbd-b928-1faa71fe7df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:01.785274 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befd37b-4641-4fbd-b928-1faa71fe7df5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:01.787486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befd37b-4641-4fbd-b928-1faa71fe7df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:01.788266 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befd37b-4641-4fbd-b928-1faa71fe7df5 admin admin] PATH_INFO: `/v3/projects/dcc7eb78e2bb4c5a9a04bd3ee9207257` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:01.792914 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0befd37b-4641-4fbd-b928-1faa71fe7df5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcc7eb78e2bb4c5a9a04bd3ee9207257)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:01.802437 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0befd37b-4641-4fbd-b928-1faa71fe7df5 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:01.930420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 886/1753] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Aug 8 08:42:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:02.009308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-0befd37b-4641-4fbd-b928-1faa71fe7df5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcc7eb78e2bb4c5a9a04bd3ee9207257', 'request_id': 'req-0befd37b-4641-4fbd-b928-1faa71fe7df5'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:02.018293 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-0befd37b-4641-4fbd-b928-1faa71fe7df5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcc7eb78e2bb4c5a9a04bd3ee9207257', 'request_id': 'req-0befd37b-4641-4fbd-b928-1faa71fe7df5'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:02.025485 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-0befd37b-4641-4fbd-b928-1faa71fe7df5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcc7eb78e2bb4c5a9a04bd3ee9207257', 'request_id': 'req-0befd37b-4641-4fbd-b928-1faa71fe7df5'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:02.074556 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99fdcb-6db9-4d5b-b68d-98123d4b5268 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:02.074556 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99fdcb-6db9-4d5b-b68d-98123d4b5268 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:02.074556 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99fdcb-6db9-4d5b-b68d-98123d4b5268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:02.127817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-6c99fdcb-6db9-4d5b-b68d-98123d4b5268 None None] MFA Rules not processed for user `32dcdc2e24e44a07ade44d1da330a77e`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:02.132154 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6c99fdcb-6db9-4d5b-b68d-98123d4b5268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:02.169358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-0befd37b-4641-4fbd-b928-1faa71fe7df5 admin admin] The token cache is being invalidate because project dcc7eb78e2bb4c5a9a04bd3ee9207257 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:42:02.170009 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-0befd37b-4641-4fbd-b928-1faa71fe7df5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:02.171068 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-0befd37b-4641-4fbd-b928-1faa71fe7df5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:02.182440 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 868/1754] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:42:01 2022] DELETE /identity/v3/projects/dcc7eb78e2bb4c5a9a04bd3ee9207257 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:42:02.184182 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befd37b-4641-4fbd-b928-1faa71fe7df5 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:02.195551 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0befd37b-4641-4fbd-b928-1faa71fe7df5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:02.229776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:02.250515 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 887/1755] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:02.253241 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c99fdcb-6db9-4d5b-b68d-98123d4b5268 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:02.255626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6c99fdcb-6db9-4d5b-b68d-98123d4b5268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:02.370644 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:02.375312 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befd37b-4641-4fbd-b928-1faa71fe7df5 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:02.400540 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80210b9d-ae2b-44d6-a994-cb9b68fab731 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:02.402086 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80210b9d-ae2b-44d6-a994-cb9b68fab731 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:02.402546 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80210b9d-ae2b-44d6-a994-cb9b68fab731 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:02.403007 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80210b9d-ae2b-44d6-a994-cb9b68fab731 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:02.430917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-80210b9d-ae2b-44d6-a994-cb9b68fab731 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:02.468395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c99fdcb-6db9-4d5b-b68d-98123d4b5268 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:02.489789 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80210b9d-ae2b-44d6-a994-cb9b68fab731 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:02.491081 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e1f82-0e5e-4e8b-b638-f3f1e71f09f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:02.491810 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e1f82-0e5e-4e8b-b638-f3f1e71f09f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:02.492132 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e1f82-0e5e-4e8b-b638-f3f1e71f09f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:02.493052 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80210b9d-ae2b-44d6-a994-cb9b68fab731 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:02.496572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e1f82-0e5e-4e8b-b638-f3f1e71f09f2 admin admin] PATH_INFO: `/v3/users/17332269b2e74c01a3a268a51424dab4` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:02.498538 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e1f82-0e5e-4e8b-b638-f3f1e71f09f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17332269b2e74c01a3a268a51424dab4)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:02.506303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e1f82-0e5e-4e8b-b638-f3f1e71f09f2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:02.576355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 869/1756] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:42:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:42:02.738081 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e98e1f82-0e5e-4e8b-b638-f3f1e71f09f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17332269b2e74c01a3a268a51424dab4'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:02.738819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e98e1f82-0e5e-4e8b-b638-f3f1e71f09f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17332269b2e74c01a3a268a51424dab4'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:02.739527 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e98e1f82-0e5e-4e8b-b638-f3f1e71f09f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17332269b2e74c01a3a268a51424dab4'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:02.753095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e98e1f82-0e5e-4e8b-b638-f3f1e71f09f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17332269b2e74c01a3a268a51424dab4'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:02.820228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e98e1f82-0e5e-4e8b-b638-f3f1e71f09f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17332269b2e74c01a3a268a51424dab4'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:02.862536 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 888/1757] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:42:02 2022] DELETE /identity/v3/users/17332269b2e74c01a3a268a51424dab4 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:03.393680 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80210b9d-ae2b-44d6-a994-cb9b68fab731 service nova] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:03.395134 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-80210b9d-ae2b-44d6-a994-cb9b68fab731 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:03.412197 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80210b9d-ae2b-44d6-a994-cb9b68fab731 service nova] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:03.559032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf9c5f-b00f-4475-aaa6-21ceef45d695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:03.567800 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf9c5f-b00f-4475-aaa6-21ceef45d695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:03.568530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf9c5f-b00f-4475-aaa6-21ceef45d695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:03.569011 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf9c5f-b00f-4475-aaa6-21ceef45d695 admin admin] PATH_INFO: `/v3/projects/bf9a5deb6e6a4eb0bbc392f8873348d7` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:03.572180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbf9c5f-b00f-4475-aaa6-21ceef45d695 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf9a5deb6e6a4eb0bbc392f8873348d7)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:03.586455 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbf9c5f-b00f-4475-aaa6-21ceef45d695 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:03.794914 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-edbf9c5f-b00f-4475-aaa6-21ceef45d695 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf9a5deb6e6a4eb0bbc392f8873348d7', 'request_id': 'req-edbf9c5f-b00f-4475-aaa6-21ceef45d695'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:03.815940 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-edbf9c5f-b00f-4475-aaa6-21ceef45d695 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf9a5deb6e6a4eb0bbc392f8873348d7', 'request_id': 'req-edbf9c5f-b00f-4475-aaa6-21ceef45d695'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:03.826591 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-edbf9c5f-b00f-4475-aaa6-21ceef45d695 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf9a5deb6e6a4eb0bbc392f8873348d7', 'request_id': 'req-edbf9c5f-b00f-4475-aaa6-21ceef45d695'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:03.943919 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-edbf9c5f-b00f-4475-aaa6-21ceef45d695 admin admin] The token cache is being invalidate because project bf9a5deb6e6a4eb0bbc392f8873348d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:42:03.944219 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-edbf9c5f-b00f-4475-aaa6-21ceef45d695 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:03.945233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-edbf9c5f-b00f-4475-aaa6-21ceef45d695 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:03.952730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 870/1758] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:42:03 2022] DELETE /identity/v3/projects/bf9a5deb6e6a4eb0bbc392f8873348d7 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:03.955635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e1f82-0e5e-4e8b-b638-f3f1e71f09f2 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:03.957655 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e98e1f82-0e5e-4e8b-b638-f3f1e71f09f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:03.998085 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e1f82-0e5e-4e8b-b638-f3f1e71f09f2 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:04.142816 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2393a1-56f4-4da3-9e3d-ba2ef78c6242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:04.145427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2393a1-56f4-4da3-9e3d-ba2ef78c6242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:04.145899 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2393a1-56f4-4da3-9e3d-ba2ef78c6242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:04.146200 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2393a1-56f4-4da3-9e3d-ba2ef78c6242 admin admin] PATH_INFO: `/v3/users/8a4635cd1e384ff397b6d5d29ef325f8` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:04.148027 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2393a1-56f4-4da3-9e3d-ba2ef78c6242 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a4635cd1e384ff397b6d5d29ef325f8)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:04.151057 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2393a1-56f4-4da3-9e3d-ba2ef78c6242 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:04.214048 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf9c5f-b00f-4475-aaa6-21ceef45d695 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:04.215607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-edbf9c5f-b00f-4475-aaa6-21ceef45d695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:04.263557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf9c5f-b00f-4475-aaa6-21ceef45d695 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:04.279390 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd17fd-31c1-4030-ac71-9962b5d95e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:04.280420 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd17fd-31c1-4030-ac71-9962b5d95e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:04.280842 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd17fd-31c1-4030-ac71-9962b5d95e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:04.281299 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd17fd-31c1-4030-ac71-9962b5d95e0b admin admin] PATH_INFO: `/v3/projects/00ac0ac7198d4c0f916d73d36aec840c` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:04.286381 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd17fd-31c1-4030-ac71-9962b5d95e0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=00ac0ac7198d4c0f916d73d36aec840c)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:04.294180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd17fd-31c1-4030-ac71-9962b5d95e0b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:04.324424 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:04.404162 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ff2393a1-56f4-4da3-9e3d-ba2ef78c6242 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a4635cd1e384ff397b6d5d29ef325f8'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:04.406390 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ff2393a1-56f4-4da3-9e3d-ba2ef78c6242 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a4635cd1e384ff397b6d5d29ef325f8'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:04.416526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ff2393a1-56f4-4da3-9e3d-ba2ef78c6242 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a4635cd1e384ff397b6d5d29ef325f8'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:04.424123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:04.424921 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ff2393a1-56f4-4da3-9e3d-ba2ef78c6242 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a4635cd1e384ff397b6d5d29ef325f8'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:04.501265 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-49dd17fd-31c1-4030-ac71-9962b5d95e0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00ac0ac7198d4c0f916d73d36aec840c', 'request_id': 'req-49dd17fd-31c1-4030-ac71-9962b5d95e0b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:04.510379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ff2393a1-56f4-4da3-9e3d-ba2ef78c6242 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a4635cd1e384ff397b6d5d29ef325f8'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:04.515245 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-49dd17fd-31c1-4030-ac71-9962b5d95e0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00ac0ac7198d4c0f916d73d36aec840c', 'request_id': 'req-49dd17fd-31c1-4030-ac71-9962b5d95e0b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:04.522889 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-49dd17fd-31c1-4030-ac71-9962b5d95e0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00ac0ac7198d4c0f916d73d36aec840c', 'request_id': 'req-49dd17fd-31c1-4030-ac71-9962b5d95e0b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:04.543048 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 889/1759] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:42:03 2022] DELETE /identity/v3/users/8a4635cd1e384ff397b6d5d29ef325f8 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:04.670755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-49dd17fd-31c1-4030-ac71-9962b5d95e0b admin admin] The token cache is being invalidate because project 00ac0ac7198d4c0f916d73d36aec840c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:42:04.671404 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-49dd17fd-31c1-4030-ac71-9962b5d95e0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:04.672307 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-49dd17fd-31c1-4030-ac71-9962b5d95e0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:04.674055 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 871/1760] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:42:04 2022] DELETE /identity/v3/projects/00ac0ac7198d4c0f916d73d36aec840c => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:04.679631 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5a9de-5ec4-4f43-bc34-c93838ba01a1 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:04.680029 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5a9de-5ec4-4f43-bc34-c93838ba01a1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:04.680303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5a9de-5ec4-4f43-bc34-c93838ba01a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:04.720418 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-83c5a9de-5ec4-4f43-bc34-c93838ba01a1 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:04.721699 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-83c5a9de-5ec4-4f43-bc34-c93838ba01a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:04.826962 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 890/1761] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:42:04.835591 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a808c4-c934-4c84-a3d1-ad97c9396767 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:04.835888 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a808c4-c934-4c84-a3d1-ad97c9396767 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:04.836341 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a808c4-c934-4c84-a3d1-ad97c9396767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:04.889913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-23a808c4-c934-4c84-a3d1-ad97c9396767 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:04.891916 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-23a808c4-c934-4c84-a3d1-ad97c9396767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:04.939065 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 872/1762] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:04.940061 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5a9de-5ec4-4f43-bc34-c93838ba01a1 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:04.954988 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5a9de-5ec4-4f43-bc34-c93838ba01a1 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:04.976450 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e7589-ce9d-40b4-b0bb-ff9c0de52c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:04.977138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e7589-ce9d-40b4-b0bb-ff9c0de52c02 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:04.977435 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e7589-ce9d-40b4-b0bb-ff9c0de52c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:04.977713 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e7589-ce9d-40b4-b0bb-ff9c0de52c02 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:04.978623 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e7589-ce9d-40b4-b0bb-ff9c0de52c02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:42:04.979741 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e7589-ce9d-40b4-b0bb-ff9c0de52c02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:04.982379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e7589-ce9d-40b4-b0bb-ff9c0de52c02 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:05.002141 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 891/1763] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:42:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:42:05.006597 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a808c4-c934-4c84-a3d1-ad97c9396767 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:05.017523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a808c4-c934-4c84-a3d1-ad97c9396767 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:05.023038 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b354bc-6827-45ed-8117-b872f00e0c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:05.023908 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b354bc-6827-45ed-8117-b872f00e0c7e admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:05.024190 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b354bc-6827-45ed-8117-b872f00e0c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:05.024461 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b354bc-6827-45ed-8117-b872f00e0c7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:05.025092 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b354bc-6827-45ed-8117-b872f00e0c7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:05.027094 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b354bc-6827-45ed-8117-b872f00e0c7e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:05.187444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 873/1764] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:42:05 2022] POST /identity/v3/projects => generated 371 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:42:05.191633 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e7589-ce9d-40b4-b0bb-ff9c0de52c02 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:05.207516 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e7589-ce9d-40b4-b0bb-ff9c0de52c02 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:05.322943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b96458-80d4-4146-8c20-cbfeeff88d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:05.323704 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b96458-80d4-4146-8c20-cbfeeff88d9a admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:05.324129 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b96458-80d4-4146-8c20-cbfeeff88d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:05.324718 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b96458-80d4-4146-8c20-cbfeeff88d9a admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:05.325354 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b96458-80d4-4146-8c20-cbfeeff88d9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:05.328277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b96458-80d4-4146-8c20-cbfeeff88d9a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:05.380992 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b354bc-6827-45ed-8117-b872f00e0c7e admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:05.383560 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-84b354bc-6827-45ed-8117-b872f00e0c7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:05.400307 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b354bc-6827-45ed-8117-b872f00e0c7e admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:05.406798 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5acb5-2160-4703-856f-8f2380ebacd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:05.407912 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5acb5-2160-4703-856f-8f2380ebacd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:05.408270 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5acb5-2160-4703-856f-8f2380ebacd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:05.408656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5acb5-2160-4703-856f-8f2380ebacd9 admin admin] PATH_INFO: `/v3/projects/3e222d375e7d4baebec0b763db8c2113` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:05.409973 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5acb5-2160-4703-856f-8f2380ebacd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e222d375e7d4baebec0b763db8c2113)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:05.411989 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5acb5-2160-4703-856f-8f2380ebacd9 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:05.446114 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 892/1765] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:42:05 2022] POST /identity/v3/users => generated 286 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:42:05.453906 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b96458-80d4-4146-8c20-cbfeeff88d9a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:05.483078 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b96458-80d4-4146-8c20-cbfeeff88d9a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:05.498488 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d499095-5b11-40d1-91ec-003a7a8e79c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:05.499619 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d499095-5b11-40d1-91ec-003a7a8e79c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:05.499897 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d499095-5b11-40d1-91ec-003a7a8e79c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:05.500175 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d499095-5b11-40d1-91ec-003a7a8e79c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:05.500815 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d499095-5b11-40d1-91ec-003a7a8e79c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:05.502770 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d499095-5b11-40d1-91ec-003a7a8e79c7 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:05.546325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 893/1766] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:42:05.547389 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d499095-5b11-40d1-91ec-003a7a8e79c7 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:05.558124 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d499095-5b11-40d1-91ec-003a7a8e79c7 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:05.566330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f67c0c-bbbc-4c7f-967d-6d3e5a02d071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:05.569281 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f67c0c-bbbc-4c7f-967d-6d3e5a02d071 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:05.569652 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f67c0c-bbbc-4c7f-967d-6d3e5a02d071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:05.571379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f67c0c-bbbc-4c7f-967d-6d3e5a02d071 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:05.571379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f67c0c-bbbc-4c7f-967d-6d3e5a02d071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:05.577456 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f67c0c-bbbc-4c7f-967d-6d3e5a02d071 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:05.617703 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 894/1767] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:42:05.622163 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f67c0c-bbbc-4c7f-967d-6d3e5a02d071 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:05.645946 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-20b5acb5-2160-4703-856f-8f2380ebacd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e222d375e7d4baebec0b763db8c2113', 'request_id': 'req-20b5acb5-2160-4703-856f-8f2380ebacd9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:05.658062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-20b5acb5-2160-4703-856f-8f2380ebacd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e222d375e7d4baebec0b763db8c2113', 'request_id': 'req-20b5acb5-2160-4703-856f-8f2380ebacd9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:05.665529 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f67c0c-bbbc-4c7f-967d-6d3e5a02d071 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:05.667795 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-20b5acb5-2160-4703-856f-8f2380ebacd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e222d375e7d4baebec0b763db8c2113', 'request_id': 'req-20b5acb5-2160-4703-856f-8f2380ebacd9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:05.707690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:05.758047 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-20b5acb5-2160-4703-856f-8f2380ebacd9 admin admin] The token cache is being invalidate because project 3e222d375e7d4baebec0b763db8c2113 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:42:05.758723 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-20b5acb5-2160-4703-856f-8f2380ebacd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:05.760031 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-20b5acb5-2160-4703-856f-8f2380ebacd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:05.762326 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 874/1768] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:42:05 2022] DELETE /identity/v3/projects/3e222d375e7d4baebec0b763db8c2113 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:05.768001 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279fe936-4859-4b68-bce0-56b325c1c9f0 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:05.768494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279fe936-4859-4b68-bce0-56b325c1c9f0 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:05.768910 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279fe936-4859-4b68-bce0-56b325c1c9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:05.798011 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-279fe936-4859-4b68-bce0-56b325c1c9f0 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:05.798692 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d7f48-c99b-441c-87ad-2ccd7541670c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:05.799384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d7f48-c99b-441c-87ad-2ccd7541670c admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:05.799810 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d7f48-c99b-441c-87ad-2ccd7541670c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:05.800694 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d7f48-c99b-441c-87ad-2ccd7541670c admin admin] PATH_INFO: `/v3/projects/876198200cb2444ca392b5cd7cbf9edb/users/4dd0fe71ad3440c6a733fcb2b0df1cc6/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:05.801777 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-279fe936-4859-4b68-bce0-56b325c1c9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:05.813644 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 875/1769] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:05.833139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598bb4b-3c3a-41b8-b88a-cdfeeef2dd79 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:05.833835 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d7f48-c99b-441c-87ad-2ccd7541670c admin admin] RBAC: Authorizing `identity:create_grant(project_id=876198200cb2444ca392b5cd7cbf9edb, user_id=4dd0fe71ad3440c6a733fcb2b0df1cc6, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:05.840132 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598bb4b-3c3a-41b8-b88a-cdfeeef2dd79 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:05.840806 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598bb4b-3c3a-41b8-b88a-cdfeeef2dd79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:05.841862 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d7f48-c99b-441c-87ad-2ccd7541670c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:05.930220 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 895/1770] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:42:05 2022] PUT /identity/v3/projects/876198200cb2444ca392b5cd7cbf9edb/users/4dd0fe71ad3440c6a733fcb2b0df1cc6/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:05.930807 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-0598bb4b-3c3a-41b8-b88a-cdfeeef2dd79 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:05.933251 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0598bb4b-3c3a-41b8-b88a-cdfeeef2dd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:05.944040 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d83415-785e-4cef-b6be-03e52bb8c948 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:05.946029 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d83415-785e-4cef-b6be-03e52bb8c948 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:05.946029 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d83415-785e-4cef-b6be-03e52bb8c948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:05.988409 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b3d83415-785e-4cef-b6be-03e52bb8c948 None None] MFA Rules not processed for user `4dd0fe71ad3440c6a733fcb2b0df1cc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:06.000038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b3d83415-785e-4cef-b6be-03e52bb8c948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:06.062628 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:06.077703 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 876/1771] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:42:06.081869 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0598bb4b-3c3a-41b8-b88a-cdfeeef2dd79 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:06.094717 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0598bb4b-3c3a-41b8-b88a-cdfeeef2dd79 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:06.104597 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f601c92-0d09-494d-9ded-b9778e42103f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:06.105508 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f601c92-0d09-494d-9ded-b9778e42103f admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:06.105909 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f601c92-0d09-494d-9ded-b9778e42103f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:06.106307 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f601c92-0d09-494d-9ded-b9778e42103f admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:06.107266 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f601c92-0d09-494d-9ded-b9778e42103f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:42:06.108026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f601c92-0d09-494d-9ded-b9778e42103f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:06.110934 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f601c92-0d09-494d-9ded-b9778e42103f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:06.129931 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 877/1772] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:42:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:42:06.134542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f601c92-0d09-494d-9ded-b9778e42103f admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:06.149309 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f601c92-0d09-494d-9ded-b9778e42103f admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:06.159493 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca131b27-6735-4c5f-bf19-fe55d53f3064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:06.160240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca131b27-6735-4c5f-bf19-fe55d53f3064 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:06.161347 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca131b27-6735-4c5f-bf19-fe55d53f3064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:06.162536 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca131b27-6735-4c5f-bf19-fe55d53f3064 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:06.166126 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca131b27-6735-4c5f-bf19-fe55d53f3064 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:06.171625 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca131b27-6735-4c5f-bf19-fe55d53f3064 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:06.180699 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 896/1773] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:06.188350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d83415-785e-4cef-b6be-03e52bb8c948 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:06.192163 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b3d83415-785e-4cef-b6be-03e52bb8c948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:06.220485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d83415-785e-4cef-b6be-03e52bb8c948 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:06.225906 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d1178-17f4-4f2c-8aac-b81b4c3ced9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:06.226618 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d1178-17f4-4f2c-8aac-b81b4c3ced9b admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:06.226949 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d1178-17f4-4f2c-8aac-b81b4c3ced9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:06.227320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d1178-17f4-4f2c-8aac-b81b4c3ced9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:06.228159 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d1178-17f4-4f2c-8aac-b81b4c3ced9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:06.230706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d1178-17f4-4f2c-8aac-b81b4c3ced9b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:06.312456 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 878/1774] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:42:06 2022] POST /identity/v3/projects => generated 399 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:42:06.316416 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca131b27-6735-4c5f-bf19-fe55d53f3064 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:06.370136 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 897/1775] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:42:06 2022] POST /identity/v3/projects => generated 373 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:42:06.371887 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d1178-17f4-4f2c-8aac-b81b4c3ced9b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:06.447008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca131b27-6735-4c5f-bf19-fe55d53f3064 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:06.513706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d1178-17f4-4f2c-8aac-b81b4c3ced9b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:06.531435 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a940c9-fbac-4967-9bac-079662ab89d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:06.532568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a940c9-fbac-4967-9bac-079662ab89d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:06.532983 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a940c9-fbac-4967-9bac-079662ab89d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:06.533425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a940c9-fbac-4967-9bac-079662ab89d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:06.534254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a940c9-fbac-4967-9bac-079662ab89d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:06.536694 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a940c9-fbac-4967-9bac-079662ab89d0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:06.637512 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a1df6-a51d-4275-ae32-1f8d8148959d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:06.638160 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a1df6-a51d-4275-ae32-1f8d8148959d admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:06.638448 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a1df6-a51d-4275-ae32-1f8d8148959d admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:06.638739 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a1df6-a51d-4275-ae32-1f8d8148959d admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:06.640112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0a1df6-a51d-4275-ae32-1f8d8148959d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:06.642289 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0a1df6-a51d-4275-ae32-1f8d8148959d admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:06.645332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 898/1776] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:42:06 2022] POST /identity/v3/users => generated 293 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:42:06.646260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a940c9-fbac-4967-9bac-079662ab89d0 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:06.675329 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a940c9-fbac-4967-9bac-079662ab89d0 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:06.681166 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127a69d-e153-4bfd-94a7-93544002ca7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:06.682108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127a69d-e153-4bfd-94a7-93544002ca7a admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:06.682392 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127a69d-e153-4bfd-94a7-93544002ca7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:06.682688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127a69d-e153-4bfd-94a7-93544002ca7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:06.683541 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d127a69d-e153-4bfd-94a7-93544002ca7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:06.687060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d127a69d-e153-4bfd-94a7-93544002ca7a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:06.688897 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:06.703843 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 899/1777] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:06 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:42:06.704960 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127a69d-e153-4bfd-94a7-93544002ca7a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:06.716301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127a69d-e153-4bfd-94a7-93544002ca7a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:06.727595 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed6dbb-58f8-4cdc-9634-a31365a8b075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:06.728355 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed6dbb-58f8-4cdc-9634-a31365a8b075 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:06.729827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed6dbb-58f8-4cdc-9634-a31365a8b075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:06.730110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed6dbb-58f8-4cdc-9634-a31365a8b075 admin admin] PATH_INFO: `/v3/projects/2788422ffa024081bf5f6dea441fa2ce/users/548242188b114dddbbbad03d2699d2c9/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:06.790848 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 879/1778] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:42:06 2022] POST /identity/v3/users => generated 300 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:42:06.796809 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ed6dbb-58f8-4cdc-9634-a31365a8b075 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2788422ffa024081bf5f6dea441fa2ce, user_id=548242188b114dddbbbad03d2699d2c9, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:06.797466 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a1df6-a51d-4275-ae32-1f8d8148959d admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:06.806790 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ed6dbb-58f8-4cdc-9634-a31365a8b075 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:06.831222 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a1df6-a51d-4275-ae32-1f8d8148959d admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:06.866901 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6dd606-30eb-43b2-9bc2-c36cf16840da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:06.867661 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dd606-30eb-43b2-9bc2-c36cf16840da admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:06.867805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dd606-30eb-43b2-9bc2-c36cf16840da admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:06.868076 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dd606-30eb-43b2-9bc2-c36cf16840da admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:06.868928 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6dd606-30eb-43b2-9bc2-c36cf16840da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:06.870643 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6dd606-30eb-43b2-9bc2-c36cf16840da admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:06.895572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 900/1779] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:42:06 2022] PUT /identity/v3/projects/2788422ffa024081bf5f6dea441fa2ce/users/548242188b114dddbbbad03d2699d2c9/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:06.899755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 880/1780] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:06 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:42:06.900024 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed6dbb-58f8-4cdc-9634-a31365a8b075 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:06.903838 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6dd606-30eb-43b2-9bc2-c36cf16840da admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:06.933564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6dd606-30eb-43b2-9bc2-c36cf16840da admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:07.080297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:07.114701 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749ca64-06b2-48eb-af6d-5efd17112cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:07.115337 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749ca64-06b2-48eb-af6d-5efd17112cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:07.115618 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749ca64-06b2-48eb-af6d-5efd17112cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:07.115895 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749ca64-06b2-48eb-af6d-5efd17112cf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:07.116548 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2749ca64-06b2-48eb-af6d-5efd17112cf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:07.118175 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2749ca64-06b2-48eb-af6d-5efd17112cf6 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:07.133126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed6dbb-58f8-4cdc-9634-a31365a8b075 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:07.149805 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef1005-9030-43ef-9546-9924527cbd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:07.150010 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 881/1781] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:06 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:42:07.150438 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef1005-9030-43ef-9546-9924527cbd99 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:07.151955 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749ca64-06b2-48eb-af6d-5efd17112cf6 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:07.155759 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef1005-9030-43ef-9546-9924527cbd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:07.156230 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef1005-9030-43ef-9546-9924527cbd99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:07.157214 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef1005-9030-43ef-9546-9924527cbd99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:07.159424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef1005-9030-43ef-9546-9924527cbd99 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:07.175494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749ca64-06b2-48eb-af6d-5efd17112cf6 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:07.187364 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e35273b-a1d5-4c99-9518-c22d133268b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:07.188120 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35273b-a1d5-4c99-9518-c22d133268b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:07.188402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35273b-a1d5-4c99-9518-c22d133268b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:07.188680 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35273b-a1d5-4c99-9518-c22d133268b4 admin admin] PATH_INFO: `/v3/projects/7f7642e3576241ef8903e4a4d9af1319/users/f6ec846040df405caa7d1a81b7bbf33f/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:07.195198 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 901/1782] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:06 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:42:07.196186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef1005-9030-43ef-9546-9924527cbd99 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:07.215033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef1005-9030-43ef-9546-9924527cbd99 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:07.228134 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b1f14-ce3e-443f-a891-2d94b05315ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:07.228134 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b1f14-ce3e-443f-a891-2d94b05315ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:07.228134 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b1f14-ce3e-443f-a891-2d94b05315ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:07.228134 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b1f14-ce3e-443f-a891-2d94b05315ff admin admin] PATH_INFO: `/v3/projects/2788422ffa024081bf5f6dea441fa2ce/users/548242188b114dddbbbad03d2699d2c9/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:07.228541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e35273b-a1d5-4c99-9518-c22d133268b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f7642e3576241ef8903e4a4d9af1319, user_id=f6ec846040df405caa7d1a81b7bbf33f, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:07.228616 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9b1f14-ce3e-443f-a891-2d94b05315ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=2788422ffa024081bf5f6dea441fa2ce, user_id=548242188b114dddbbbad03d2699d2c9, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:07.228616 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9b1f14-ce3e-443f-a891-2d94b05315ff admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:07.234414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e35273b-a1d5-4c99-9518-c22d133268b4 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:07.245749 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 902/1783] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:42:07 2022] PUT /identity/v3/projects/2788422ffa024081bf5f6dea441fa2ce/users/548242188b114dddbbbad03d2699d2c9/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:07.260784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90219b-d0b3-4af4-83f2-d9668ba0794b None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:07.263587 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90219b-d0b3-4af4-83f2-d9668ba0794b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:07.263954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90219b-d0b3-4af4-83f2-d9668ba0794b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:07.324115 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-6e90219b-d0b3-4af4-83f2-d9668ba0794b None None] MFA Rules not processed for user `548242188b114dddbbbad03d2699d2c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:07.326560 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6e90219b-d0b3-4af4-83f2-d9668ba0794b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:07.353904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 882/1784] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:42:07 2022] PUT /identity/v3/projects/7f7642e3576241ef8903e4a4d9af1319/users/f6ec846040df405caa7d1a81b7bbf33f/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:07.366982 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a94c8d-a4d2-4517-8b22-f62f21bcf633 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:07.367826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a94c8d-a4d2-4517-8b22-f62f21bcf633 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:07.368604 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a94c8d-a4d2-4517-8b22-f62f21bcf633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:07.437585 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-80a94c8d-a4d2-4517-8b22-f62f21bcf633 None None] MFA Rules not processed for user `f6ec846040df405caa7d1a81b7bbf33f`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:07.438847 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-80a94c8d-a4d2-4517-8b22-f62f21bcf633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:07.485008 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 903/1785] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:07.500559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90219b-d0b3-4af4-83f2-d9668ba0794b None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:07.502518 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6e90219b-d0b3-4af4-83f2-d9668ba0794b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:07.570536 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 883/1786] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:07 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:42:07.590010 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a94c8d-a4d2-4517-8b22-f62f21bcf633 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:07.618825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:07.638484 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90219b-d0b3-4af4-83f2-d9668ba0794b None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:07.728535 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a94c8d-a4d2-4517-8b22-f62f21bcf633 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:07.736098 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c01f3c-da0c-4371-ae14-a128db2dfdec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:07.736845 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c01f3c-da0c-4371-ae14-a128db2dfdec admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:07.737075 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c01f3c-da0c-4371-ae14-a128db2dfdec admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:07.737406 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c01f3c-da0c-4371-ae14-a128db2dfdec admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:07.738050 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c01f3c-da0c-4371-ae14-a128db2dfdec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:07.740456 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c01f3c-da0c-4371-ae14-a128db2dfdec admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:07.752667 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65798963-ce72-40cb-84dc-84d9c453b904 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:07.752944 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65798963-ce72-40cb-84dc-84d9c453b904 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:07.753293 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65798963-ce72-40cb-84dc-84d9c453b904 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:07.753430 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65798963-ce72-40cb-84dc-84d9c453b904 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:07.868190 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 884/1787] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:42:07 2022] POST /identity/v3/projects => generated 399 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:42:07.872710 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c01f3c-da0c-4371-ae14-a128db2dfdec admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:07.912042 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65798963-ce72-40cb-84dc-84d9c453b904 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:07.914880 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65798963-ce72-40cb-84dc-84d9c453b904 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:07.934967 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 904/1788] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:42:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:42:07.975631 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:08.036713 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c01f3c-da0c-4371-ae14-a128db2dfdec admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:08.053271 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0feaf4-9f9b-4444-b455-7ca1999346a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:08.053917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0feaf4-9f9b-4444-b455-7ca1999346a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:08.054199 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0feaf4-9f9b-4444-b455-7ca1999346a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:08.054476 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0feaf4-9f9b-4444-b455-7ca1999346a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:08.055264 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0feaf4-9f9b-4444-b455-7ca1999346a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:08.059219 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0feaf4-9f9b-4444-b455-7ca1999346a2 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:08.204391 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 885/1789] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:42:07 2022] POST /identity/v3/users => generated 306 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:42:08.208398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65798963-ce72-40cb-84dc-84d9c453b904 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:08.231806 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65798963-ce72-40cb-84dc-84d9c453b904 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:08.254370 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0feaf4-9f9b-4444-b455-7ca1999346a2 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:08.257429 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812eae21-7609-4ec7-96a7-f0b1dd552e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:08.258208 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812eae21-7609-4ec7-96a7-f0b1dd552e26 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:08.258550 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812eae21-7609-4ec7-96a7-f0b1dd552e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:08.261672 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812eae21-7609-4ec7-96a7-f0b1dd552e26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:08.261672 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812eae21-7609-4ec7-96a7-f0b1dd552e26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:08.267286 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812eae21-7609-4ec7-96a7-f0b1dd552e26 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:08.306268 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 905/1790] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:42:08.312435 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812eae21-7609-4ec7-96a7-f0b1dd552e26 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:08.340811 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812eae21-7609-4ec7-96a7-f0b1dd552e26 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:08.361755 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00399aa4-017e-4c6a-b7f9-33bb98b99618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:08.363605 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00399aa4-017e-4c6a-b7f9-33bb98b99618 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:08.365480 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00399aa4-017e-4c6a-b7f9-33bb98b99618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:08.366598 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00399aa4-017e-4c6a-b7f9-33bb98b99618 admin admin] PATH_INFO: `/v3/projects/5ab9bfbceeaa495c92f88a6d5a82a94c/users/218ca497ac5c4b3ba59dcfcceff0f7c5/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:08.397055 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0feaf4-9f9b-4444-b455-7ca1999346a2 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:08.430347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00399aa4-017e-4c6a-b7f9-33bb98b99618 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ab9bfbceeaa495c92f88a6d5a82a94c, user_id=218ca497ac5c4b3ba59dcfcceff0f7c5, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:08.433231 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00399aa4-017e-4c6a-b7f9-33bb98b99618 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:08.439535 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f2573-ba37-4e44-b836-aea29fcce27f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:08.440296 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f2573-ba37-4e44-b836-aea29fcce27f service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:08.440680 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f2573-ba37-4e44-b836-aea29fcce27f service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:08.441015 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f2573-ba37-4e44-b836-aea29fcce27f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:08.510662 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 906/1791] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:42:08 2022] PUT /identity/v3/projects/5ab9bfbceeaa495c92f88a6d5a82a94c/users/218ca497ac5c4b3ba59dcfcceff0f7c5/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:08.518990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00399aa4-017e-4c6a-b7f9-33bb98b99618 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:08.607339 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f2573-ba37-4e44-b836-aea29fcce27f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:08.612311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f2573-ba37-4e44-b836-aea29fcce27f service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:08.638684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 886/1792] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:42:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:42:08.757168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00399aa4-017e-4c6a-b7f9-33bb98b99618 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:08.777605 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2bf4c-e4d6-498b-a507-598ece8b702d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:08.778348 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2bf4c-e4d6-498b-a507-598ece8b702d admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:08.778628 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2bf4c-e4d6-498b-a507-598ece8b702d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:08.778920 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2bf4c-e4d6-498b-a507-598ece8b702d admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:08.779671 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2bf4c-e4d6-498b-a507-598ece8b702d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:08.782497 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2bf4c-e4d6-498b-a507-598ece8b702d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:08.825107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 907/1793] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:42:08.831035 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f2573-ba37-4e44-b836-aea29fcce27f service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:08.853236 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f2573-ba37-4e44-b836-aea29fcce27f service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:08.869416 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f35dc-869c-499c-9ab6-1fa7c2839d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:08.870499 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f35dc-869c-499c-9ab6-1fa7c2839d3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:08.871766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f35dc-869c-499c-9ab6-1fa7c2839d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:08.872230 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f35dc-869c-499c-9ab6-1fa7c2839d3e admin admin] PATH_INFO: `/v3/projects/5ab9bfbceeaa495c92f88a6d5a82a94c/users/218ca497ac5c4b3ba59dcfcceff0f7c5/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:08.880647 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f35dc-869c-499c-9ab6-1fa7c2839d3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ab9bfbceeaa495c92f88a6d5a82a94c, user_id=218ca497ac5c4b3ba59dcfcceff0f7c5, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:08.883397 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f35dc-869c-499c-9ab6-1fa7c2839d3e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:08.918794 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 887/1794] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:42:08 2022] PUT /identity/v3/projects/5ab9bfbceeaa495c92f88a6d5a82a94c/users/218ca497ac5c4b3ba59dcfcceff0f7c5/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:42:08.934706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1f168-118b-48ae-be57-0ce6d3b1928a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:08.937891 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1f168-118b-48ae-be57-0ce6d3b1928a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:08.938281 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1f168-118b-48ae-be57-0ce6d3b1928a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:08.997752 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:09.061819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-1ec1f168-118b-48ae-be57-0ce6d3b1928a None None] MFA Rules not processed for user `218ca497ac5c4b3ba59dcfcceff0f7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:09.064601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1ec1f168-118b-48ae-be57-0ce6d3b1928a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:09.281664 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9eed4-6da2-4145-b8c7-8d39754cf81a None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:09.281664 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9eed4-6da2-4145-b8c7-8d39754cf81a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:09.281664 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9eed4-6da2-4145-b8c7-8d39754cf81a None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:09.284685 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 888/1795] 10.222.0.33 () {54 vars in 921 bytes} [Mon Aug 8 08:42:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 08:42:09.341306 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda0660-6c66-4695-825f-ba0f45ef8ced None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:09.341730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda0660-6c66-4695-825f-ba0f45ef8ced None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:09.341994 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda0660-6c66-4695-825f-ba0f45ef8ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:09.439003 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-4eda0660-6c66-4695-825f-ba0f45ef8ced None None] MFA Rules not processed for user `a3317f751ec74bde95ec262394001292`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:09.442950 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-4eda0660-6c66-4695-825f-ba0f45ef8ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:09.480300 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 908/1796] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:08 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 550 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:09.496692 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1f168-118b-48ae-be57-0ce6d3b1928a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:09.658592 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:09.693816 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1f168-118b-48ae-be57-0ce6d3b1928a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:09.704624 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf95885-7b90-4e5a-9f86-d3ce9558ecd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:09.706779 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf95885-7b90-4e5a-9f86-d3ce9558ecd4 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:09.707068 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf95885-7b90-4e5a-9f86-d3ce9558ecd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:09.707347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf95885-7b90-4e5a-9f86-d3ce9558ecd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:09.708552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 889/1797] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Aug 8 08:42:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:09.709638 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda0660-6c66-4695-825f-ba0f45ef8ced None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:09.711883 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-4eda0660-6c66-4695-825f-ba0f45ef8ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:09.732245 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf95885-7b90-4e5a-9f86-d3ce9558ecd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:09.734986 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf95885-7b90-4e5a-9f86-d3ce9558ecd4 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:09.746936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 909/1798] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:42:09 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:42:09.830772 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda0660-6c66-4695-825f-ba0f45ef8ced None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:09.861013 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea4e07-1a73-4dac-9758-ae8b1df183d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:09.862016 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea4e07-1a73-4dac-9758-ae8b1df183d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:09.862294 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea4e07-1a73-4dac-9758-ae8b1df183d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:09.862566 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea4e07-1a73-4dac-9758-ae8b1df183d7 admin admin] PATH_INFO: `/v3/users/46aac537c3f34c0c8471f1ca6239f1a0` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:09.865781 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea4e07-1a73-4dac-9758-ae8b1df183d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46aac537c3f34c0c8471f1ca6239f1a0)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:09.867624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea4e07-1a73-4dac-9758-ae8b1df183d7 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:10.101472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf95885-7b90-4e5a-9f86-d3ce9558ecd4 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:10.103237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2bf95885-7b90-4e5a-9f86-d3ce9558ecd4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:10.119618 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf95885-7b90-4e5a-9f86-d3ce9558ecd4 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:10.127021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a3dad-6fd2-4f1d-87ac-16919b0aefc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:10.127653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a3dad-6fd2-4f1d-87ac-16919b0aefc1 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:10.127938 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a3dad-6fd2-4f1d-87ac-16919b0aefc1 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:10.128217 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a3dad-6fd2-4f1d-87ac-16919b0aefc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:10.138449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a3dad-6fd2-4f1d-87ac-16919b0aefc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:10.141256 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a3dad-6fd2-4f1d-87ac-16919b0aefc1 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:10.153574 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 910/1799] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:42:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:42:10.282658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-49ea4e07-1a73-4dac-9758-ae8b1df183d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46aac537c3f34c0c8471f1ca6239f1a0'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:10.283925 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-49ea4e07-1a73-4dac-9758-ae8b1df183d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46aac537c3f34c0c8471f1ca6239f1a0'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:10.284553 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-49ea4e07-1a73-4dac-9758-ae8b1df183d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46aac537c3f34c0c8471f1ca6239f1a0'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:10.292188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-49ea4e07-1a73-4dac-9758-ae8b1df183d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46aac537c3f34c0c8471f1ca6239f1a0'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:10.370607 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a3dad-6fd2-4f1d-87ac-16919b0aefc1 service nova] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:10.379397 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-49ea4e07-1a73-4dac-9758-ae8b1df183d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46aac537c3f34c0c8471f1ca6239f1a0'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:10.380752 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a3dad-6fd2-4f1d-87ac-16919b0aefc1 service nova] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:10.385177 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f621c040-3cc0-4474-a6b6-e049f58affdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:10.385870 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f621c040-3cc0-4474-a6b6-e049f58affdc service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:10.386146 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f621c040-3cc0-4474-a6b6-e049f58affdc service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:10.386602 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f621c040-3cc0-4474-a6b6-e049f58affdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:10.388395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f621c040-3cc0-4474-a6b6-e049f58affdc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:10.408308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 890/1800] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:42:09 2022] DELETE /identity/v3/users/46aac537c3f34c0c8471f1ca6239f1a0 => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:10.410066 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad5d81-e9f1-4ebe-b951-aa5472c5bffc None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:10.410465 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad5d81-e9f1-4ebe-b951-aa5472c5bffc None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:10.411027 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad5d81-e9f1-4ebe-b951-aa5472c5bffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:10.443400 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-06ad5d81-e9f1-4ebe-b951-aa5472c5bffc None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:10.444787 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-06ad5d81-e9f1-4ebe-b951-aa5472c5bffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:10.528775 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 891/1801] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:42:10.533870 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f621c040-3cc0-4474-a6b6-e049f58affdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:10.536415 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f621c040-3cc0-4474-a6b6-e049f58affdc service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:10.548736 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 911/1802] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:42:10 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:42:10.568338 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad5d81-e9f1-4ebe-b951-aa5472c5bffc None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:10.571494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-06ad5d81-e9f1-4ebe-b951-aa5472c5bffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:10.600356 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:10.673575 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad5d81-e9f1-4ebe-b951-aa5472c5bffc None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:10.677967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68250bfe-9b75-4d3a-a973-6b54e1652075 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:10.679364 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68250bfe-9b75-4d3a-a973-6b54e1652075 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:10.679676 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68250bfe-9b75-4d3a-a973-6b54e1652075 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:10.679955 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68250bfe-9b75-4d3a-a973-6b54e1652075 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:10.693523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68250bfe-9b75-4d3a-a973-6b54e1652075 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:10.695735 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68250bfe-9b75-4d3a-a973-6b54e1652075 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:10.708854 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 892/1803] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:42:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:42:11.721978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f621c040-3cc0-4474-a6b6-e049f58affdc service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:11.723800 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f621c040-3cc0-4474-a6b6-e049f58affdc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:11.743118 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f621c040-3cc0-4474-a6b6-e049f58affdc service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:11.762834 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43689e18-cbe0-4045-8df7-c08fb5e47988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:11.763790 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43689e18-cbe0-4045-8df7-c08fb5e47988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:11.764691 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43689e18-cbe0-4045-8df7-c08fb5e47988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:11.765020 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43689e18-cbe0-4045-8df7-c08fb5e47988 admin admin] PATH_INFO: `/v3/projects/6b06295b5f4e4e0abc0da9f00be58540` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:11.766267 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43689e18-cbe0-4045-8df7-c08fb5e47988 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b06295b5f4e4e0abc0da9f00be58540)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:11.768193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43689e18-cbe0-4045-8df7-c08fb5e47988 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:11.786574 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:11.905640 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-43689e18-cbe0-4045-8df7-c08fb5e47988 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b06295b5f4e4e0abc0da9f00be58540', 'request_id': 'req-43689e18-cbe0-4045-8df7-c08fb5e47988'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:11.973090 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-43689e18-cbe0-4045-8df7-c08fb5e47988 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b06295b5f4e4e0abc0da9f00be58540', 'request_id': 'req-43689e18-cbe0-4045-8df7-c08fb5e47988'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:11.981438 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-43689e18-cbe0-4045-8df7-c08fb5e47988 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b06295b5f4e4e0abc0da9f00be58540', 'request_id': 'req-43689e18-cbe0-4045-8df7-c08fb5e47988'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:11.993954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-43689e18-cbe0-4045-8df7-c08fb5e47988 admin admin] The token cache is being invalidate because project 6b06295b5f4e4e0abc0da9f00be58540 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:42:11.994243 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-43689e18-cbe0-4045-8df7-c08fb5e47988 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:11.995165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-43689e18-cbe0-4045-8df7-c08fb5e47988 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:11.996681 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 912/1804] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:42:11 2022] DELETE /identity/v3/projects/6b06295b5f4e4e0abc0da9f00be58540 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:12.001856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68250bfe-9b75-4d3a-a973-6b54e1652075 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:12.012126 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68250bfe-9b75-4d3a-a973-6b54e1652075 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:12.083472 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62107695-c2a6-4a81-9a3f-3831079c6373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:12.084311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62107695-c2a6-4a81-9a3f-3831079c6373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:12.084668 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62107695-c2a6-4a81-9a3f-3831079c6373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:12.085017 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62107695-c2a6-4a81-9a3f-3831079c6373 admin admin] PATH_INFO: `/v3/users/fed465dfa15444f8bc59db202a19f321` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:12.086501 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62107695-c2a6-4a81-9a3f-3831079c6373 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fed465dfa15444f8bc59db202a19f321)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:12.090203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62107695-c2a6-4a81-9a3f-3831079c6373 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:12.334175 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-62107695-c2a6-4a81-9a3f-3831079c6373 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fed465dfa15444f8bc59db202a19f321'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:12.334816 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-62107695-c2a6-4a81-9a3f-3831079c6373 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fed465dfa15444f8bc59db202a19f321'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:12.335877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-62107695-c2a6-4a81-9a3f-3831079c6373 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fed465dfa15444f8bc59db202a19f321'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:12.345704 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-62107695-c2a6-4a81-9a3f-3831079c6373 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fed465dfa15444f8bc59db202a19f321'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:12.442307 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-62107695-c2a6-4a81-9a3f-3831079c6373 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fed465dfa15444f8bc59db202a19f321'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:12.464005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 893/1805] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:42:12 2022] DELETE /identity/v3/users/fed465dfa15444f8bc59db202a19f321 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:42:13.303160 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43689e18-cbe0-4045-8df7-c08fb5e47988 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:13.306922 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-43689e18-cbe0-4045-8df7-c08fb5e47988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:13.363957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43689e18-cbe0-4045-8df7-c08fb5e47988 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:13.472879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66b5b1-a9a3-4a1c-b2a2-48c78d9108eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:13.472879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66b5b1-a9a3-4a1c-b2a2-48c78d9108eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:13.472879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66b5b1-a9a3-4a1c-b2a2-48c78d9108eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:13.472879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66b5b1-a9a3-4a1c-b2a2-48c78d9108eb admin admin] PATH_INFO: `/v3/projects/a7a1fbb085ae46348530bd3d6b764979` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:13.473166 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66b5b1-a9a3-4a1c-b2a2-48c78d9108eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7a1fbb085ae46348530bd3d6b764979)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:13.476194 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66b5b1-a9a3-4a1c-b2a2-48c78d9108eb admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:13.637851 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0c66b5b1-a9a3-4a1c-b2a2-48c78d9108eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7a1fbb085ae46348530bd3d6b764979', 'request_id': 'req-0c66b5b1-a9a3-4a1c-b2a2-48c78d9108eb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:13.698259 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0c66b5b1-a9a3-4a1c-b2a2-48c78d9108eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7a1fbb085ae46348530bd3d6b764979', 'request_id': 'req-0c66b5b1-a9a3-4a1c-b2a2-48c78d9108eb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:13.705684 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0c66b5b1-a9a3-4a1c-b2a2-48c78d9108eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7a1fbb085ae46348530bd3d6b764979', 'request_id': 'req-0c66b5b1-a9a3-4a1c-b2a2-48c78d9108eb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:13.716503 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0c66b5b1-a9a3-4a1c-b2a2-48c78d9108eb admin admin] The token cache is being invalidate because project a7a1fbb085ae46348530bd3d6b764979 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:42:13.716864 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0c66b5b1-a9a3-4a1c-b2a2-48c78d9108eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:13.718215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0c66b5b1-a9a3-4a1c-b2a2-48c78d9108eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:13.719957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 913/1806] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:42:13 2022] DELETE /identity/v3/projects/a7a1fbb085ae46348530bd3d6b764979 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:13.724961 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c0fe9-c752-42e7-be98-a348d4cd2193 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:13.728390 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c0fe9-c752-42e7-be98-a348d4cd2193 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:13.728390 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c0fe9-c752-42e7-be98-a348d4cd2193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:13.747309 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-d46c0fe9-c752-42e7-be98-a348d4cd2193 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:13.749608 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d46c0fe9-c752-42e7-be98-a348d4cd2193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:13.784548 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:13.810054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 894/1807] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:13.819560 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b419e0-3935-4678-9a9f-2e39265e86fb None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:13.819848 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b419e0-3935-4678-9a9f-2e39265e86fb None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:13.820128 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b419e0-3935-4678-9a9f-2e39265e86fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:13.839696 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-91b419e0-3935-4678-9a9f-2e39265e86fb None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:13.840905 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-91b419e0-3935-4678-9a9f-2e39265e86fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:13.850445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 914/1808] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:13.854721 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c0fe9-c752-42e7-be98-a348d4cd2193 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:13.865623 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c0fe9-c752-42e7-be98-a348d4cd2193 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:13.870319 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c58c36-25a5-4da2-b5fd-5f2091dea3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:13.870942 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c58c36-25a5-4da2-b5fd-5f2091dea3cf admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:13.871202 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c58c36-25a5-4da2-b5fd-5f2091dea3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:13.871476 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c58c36-25a5-4da2-b5fd-5f2091dea3cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:13.872127 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c58c36-25a5-4da2-b5fd-5f2091dea3cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:42:13.872428 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c58c36-25a5-4da2-b5fd-5f2091dea3cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:13.874484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c58c36-25a5-4da2-b5fd-5f2091dea3cf admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:13.889378 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 895/1809] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:42:13.890502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b419e0-3935-4678-9a9f-2e39265e86fb None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:13.900017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b419e0-3935-4678-9a9f-2e39265e86fb None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:13.905252 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf05d4-a4a6-4b4f-a642-961b1b22f546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:13.905877 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf05d4-a4a6-4b4f-a642-961b1b22f546 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:13.906191 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf05d4-a4a6-4b4f-a642-961b1b22f546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:13.906420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf05d4-a4a6-4b4f-a642-961b1b22f546 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:13.907048 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbf05d4-a4a6-4b4f-a642-961b1b22f546 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:13.909016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbf05d4-a4a6-4b4f-a642-961b1b22f546 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:14.027506 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 915/1810] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:42:13 2022] POST /identity/v3/projects => generated 367 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:42:14.032465 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c58c36-25a5-4da2-b5fd-5f2091dea3cf admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:14.043080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c58c36-25a5-4da2-b5fd-5f2091dea3cf admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:14.100082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e6c51-dafb-4b71-8bcb-30b874b25bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:14.100695 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e6c51-dafb-4b71-8bcb-30b874b25bd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:14.101037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e6c51-dafb-4b71-8bcb-30b874b25bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:14.101337 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e6c51-dafb-4b71-8bcb-30b874b25bd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:14.101997 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1e6c51-dafb-4b71-8bcb-30b874b25bd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:14.103947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1e6c51-dafb-4b71-8bcb-30b874b25bd8 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:14.203231 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 896/1811] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:42:14 2022] POST /identity/v3/users => generated 284 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:42:14.204931 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf05d4-a4a6-4b4f-a642-961b1b22f546 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:14.211677 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:14.215318 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf05d4-a4a6-4b4f-a642-961b1b22f546 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:14.220476 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5f9a8-94f6-4d1c-9411-daf36782aaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:14.221070 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5f9a8-94f6-4d1c-9411-daf36782aaf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:14.221372 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5f9a8-94f6-4d1c-9411-daf36782aaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:14.221640 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5f9a8-94f6-4d1c-9411-daf36782aaf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:14.222574 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d5f9a8-94f6-4d1c-9411-daf36782aaf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:14.224151 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d5f9a8-94f6-4d1c-9411-daf36782aaf5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:14.233991 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 916/1812] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:42:14.238066 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e6c51-dafb-4b71-8bcb-30b874b25bd8 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:14.248754 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e6c51-dafb-4b71-8bcb-30b874b25bd8 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:14.253912 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab97e48-0fbf-4a39-99fc-b72a4259ff90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:14.254637 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab97e48-0fbf-4a39-99fc-b72a4259ff90 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:14.254938 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab97e48-0fbf-4a39-99fc-b72a4259ff90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:14.255186 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab97e48-0fbf-4a39-99fc-b72a4259ff90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:14.255918 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab97e48-0fbf-4a39-99fc-b72a4259ff90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:14.257635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab97e48-0fbf-4a39-99fc-b72a4259ff90 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:14.270779 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 897/1813] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:42:14.275146 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5f9a8-94f6-4d1c-9411-daf36782aaf5 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:14.285790 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5f9a8-94f6-4d1c-9411-daf36782aaf5 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:14.294617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2412a9a6-858c-4085-a646-da5f0bc8c3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:14.294617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412a9a6-858c-4085-a646-da5f0bc8c3ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:14.294617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412a9a6-858c-4085-a646-da5f0bc8c3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:14.294617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412a9a6-858c-4085-a646-da5f0bc8c3ec admin admin] PATH_INFO: `/v3/projects/ca71fb3a1e294249acf42196e205a258/users/f395526c0c1847da8be96c7d0a8bcd58/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:14.305688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2412a9a6-858c-4085-a646-da5f0bc8c3ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca71fb3a1e294249acf42196e205a258, user_id=f395526c0c1847da8be96c7d0a8bcd58, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:14.308336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2412a9a6-858c-4085-a646-da5f0bc8c3ec admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:14.378715 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 917/1814] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:42:14 2022] PUT /identity/v3/projects/ca71fb3a1e294249acf42196e205a258/users/f395526c0c1847da8be96c7d0a8bcd58/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:14.388943 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d657295-2586-4d87-9011-69837290848d None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:14.389887 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d657295-2586-4d87-9011-69837290848d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:14.390379 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d657295-2586-4d87-9011-69837290848d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:14.421861 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-2d657295-2586-4d87-9011-69837290848d None None] MFA Rules not processed for user `f395526c0c1847da8be96c7d0a8bcd58`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:14.423279 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2d657295-2586-4d87-9011-69837290848d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:14.491552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 898/1815] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:14.504528 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2412a9a6-858c-4085-a646-da5f0bc8c3ec admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:14.581675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2412a9a6-858c-4085-a646-da5f0bc8c3ec admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:14.586408 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e4ac09-d5ca-461f-8242-24fef4155613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:14.587016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4ac09-d5ca-461f-8242-24fef4155613 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:14.587295 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4ac09-d5ca-461f-8242-24fef4155613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:14.587565 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4ac09-d5ca-461f-8242-24fef4155613 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:14.588197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e4ac09-d5ca-461f-8242-24fef4155613 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:14.590182 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e4ac09-d5ca-461f-8242-24fef4155613 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:14.712985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 918/1816] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:42:14 2022] POST /identity/v3/projects => generated 369 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:42:14.714425 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d657295-2586-4d87-9011-69837290848d None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:14.727021 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d657295-2586-4d87-9011-69837290848d None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:14.802349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a8845-f806-49c7-bdb8-ad4d7ddc5df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:14.802897 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a8845-f806-49c7-bdb8-ad4d7ddc5df8 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:14.803176 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a8845-f806-49c7-bdb8-ad4d7ddc5df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:14.803437 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a8845-f806-49c7-bdb8-ad4d7ddc5df8 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:14.804071 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a8845-f806-49c7-bdb8-ad4d7ddc5df8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:14.806085 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a8845-f806-49c7-bdb8-ad4d7ddc5df8 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:14.869126 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:14.904306 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 899/1817] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:42:14 2022] POST /identity/v3/users => generated 291 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:42:14.908851 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e4ac09-d5ca-461f-8242-24fef4155613 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:14.918823 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e4ac09-d5ca-461f-8242-24fef4155613 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:14.923368 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f746c-d435-4c81-a639-d2e37431c59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:14.923965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f746c-d435-4c81-a639-d2e37431c59f admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:14.924243 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f746c-d435-4c81-a639-d2e37431c59f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:14.924521 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f746c-d435-4c81-a639-d2e37431c59f admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:14.925248 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f746c-d435-4c81-a639-d2e37431c59f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:14.926789 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f746c-d435-4c81-a639-d2e37431c59f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:14.937280 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 919/1818] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:42:14.941772 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a8845-f806-49c7-bdb8-ad4d7ddc5df8 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:14.951518 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a8845-f806-49c7-bdb8-ad4d7ddc5df8 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:14.956766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d64e85-cc66-4a8a-ad3e-b89a98901f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:14.957448 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d64e85-cc66-4a8a-ad3e-b89a98901f20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:14.957723 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d64e85-cc66-4a8a-ad3e-b89a98901f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:14.957992 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d64e85-cc66-4a8a-ad3e-b89a98901f20 admin admin] PATH_INFO: `/v3/projects/e862bba3857c4a30b381968d32d02bab/users/22b810dbed6144c89dbcc3cee9e725a4/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:14.972977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d64e85-cc66-4a8a-ad3e-b89a98901f20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e862bba3857c4a30b381968d32d02bab, user_id=22b810dbed6144c89dbcc3cee9e725a4, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:14.975624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d64e85-cc66-4a8a-ad3e-b89a98901f20 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:15.995852 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 900/1819] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:42:14 2022] PUT /identity/v3/projects/e862bba3857c4a30b381968d32d02bab/users/22b810dbed6144c89dbcc3cee9e725a4/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 1055 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:16.000149 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f746c-d435-4c81-a639-d2e37431c59f admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:16.010324 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f746c-d435-4c81-a639-d2e37431c59f admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:16.077228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd58f96-7dd5-401b-b93a-59cb655988b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:16.077228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd58f96-7dd5-401b-b93a-59cb655988b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:16.077228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd58f96-7dd5-401b-b93a-59cb655988b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:16.077228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd58f96-7dd5-401b-b93a-59cb655988b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:16.077617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd58f96-7dd5-401b-b93a-59cb655988b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:16.077617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd58f96-7dd5-401b-b93a-59cb655988b4 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:16.086173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 920/1820] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:42:16.090355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d64e85-cc66-4a8a-ad3e-b89a98901f20 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:16.101813 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d64e85-cc66-4a8a-ad3e-b89a98901f20 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:16.106819 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107a6ea-c664-4e72-83cc-ab7d4aabd3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:16.107478 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107a6ea-c664-4e72-83cc-ab7d4aabd3f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:16.107764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107a6ea-c664-4e72-83cc-ab7d4aabd3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:16.108123 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107a6ea-c664-4e72-83cc-ab7d4aabd3f3 admin admin] PATH_INFO: `/v3/projects/e862bba3857c4a30b381968d32d02bab/users/22b810dbed6144c89dbcc3cee9e725a4/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:16.110371 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107a6ea-c664-4e72-83cc-ab7d4aabd3f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e862bba3857c4a30b381968d32d02bab, user_id=22b810dbed6144c89dbcc3cee9e725a4, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:16.113158 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107a6ea-c664-4e72-83cc-ab7d4aabd3f3 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:16.124987 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 901/1821] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:42:16 2022] PUT /identity/v3/projects/e862bba3857c4a30b381968d32d02bab/users/22b810dbed6144c89dbcc3cee9e725a4/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:16.135393 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbad815-6d02-4892-8558-d28958790086 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:16.135743 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbad815-6d02-4892-8558-d28958790086 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:16.140354 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbad815-6d02-4892-8558-d28958790086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:16.164103 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-2fbad815-6d02-4892-8558-d28958790086 None None] MFA Rules not processed for user `22b810dbed6144c89dbcc3cee9e725a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:16.165339 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2fbad815-6d02-4892-8558-d28958790086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:16.195331 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:16.234416 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 921/1822] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:16 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:16.245341 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107a6ea-c664-4e72-83cc-ab7d4aabd3f3 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:16.247005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f107a6ea-c664-4e72-83cc-ab7d4aabd3f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:16.309463 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107a6ea-c664-4e72-83cc-ab7d4aabd3f3 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:16.314293 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb461a0a-71e4-4e18-8a24-ca47acb88978 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:16.314948 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb461a0a-71e4-4e18-8a24-ca47acb88978 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:16.315301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb461a0a-71e4-4e18-8a24-ca47acb88978 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:16.315481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb461a0a-71e4-4e18-8a24-ca47acb88978 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:16.325795 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb461a0a-71e4-4e18-8a24-ca47acb88978 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:16.327516 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb461a0a-71e4-4e18-8a24-ca47acb88978 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:16.339338 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 902/1823] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:42:16 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:42:17.684238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbad815-6d02-4892-8558-d28958790086 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:17.687196 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2fbad815-6d02-4892-8558-d28958790086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:17.755164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbad815-6d02-4892-8558-d28958790086 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:17.759729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ad126-8fd5-46cb-9065-4a2e16467eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:17.760370 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ad126-8fd5-46cb-9065-4a2e16467eee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:17.760768 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ad126-8fd5-46cb-9065-4a2e16467eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:17.760927 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ad126-8fd5-46cb-9065-4a2e16467eee admin admin] PATH_INFO: `/v3/users/f6ec846040df405caa7d1a81b7bbf33f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:17.762092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539ad126-8fd5-46cb-9065-4a2e16467eee admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6ec846040df405caa7d1a81b7bbf33f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:17.764582 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539ad126-8fd5-46cb-9065-4a2e16467eee admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:17.944084 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-539ad126-8fd5-46cb-9065-4a2e16467eee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6ec846040df405caa7d1a81b7bbf33f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:17.944771 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-539ad126-8fd5-46cb-9065-4a2e16467eee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6ec846040df405caa7d1a81b7bbf33f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:17.945404 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-539ad126-8fd5-46cb-9065-4a2e16467eee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6ec846040df405caa7d1a81b7bbf33f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:17.954488 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-539ad126-8fd5-46cb-9065-4a2e16467eee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6ec846040df405caa7d1a81b7bbf33f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:18.057058 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-539ad126-8fd5-46cb-9065-4a2e16467eee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6ec846040df405caa7d1a81b7bbf33f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:18.094757 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 922/1824] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:42:17 2022] DELETE /identity/v3/users/f6ec846040df405caa7d1a81b7bbf33f => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:18.096207 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf7d5f-35c2-4887-a178-d9d99a7a30b5 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:18.096645 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf7d5f-35c2-4887-a178-d9d99a7a30b5 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:18.096754 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf7d5f-35c2-4887-a178-d9d99a7a30b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:18.120368 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-aacf7d5f-35c2-4887-a178-d9d99a7a30b5 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:18.121961 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-aacf7d5f-35c2-4887-a178-d9d99a7a30b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:18.200292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 903/1825] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:42:18.212354 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ad126-8fd5-46cb-9065-4a2e16467eee admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:18.214230 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-539ad126-8fd5-46cb-9065-4a2e16467eee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:18.334846 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:18.351704 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ad126-8fd5-46cb-9065-4a2e16467eee admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:18.359087 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e696c7-6440-4a7f-b1f3-0c56fb1b3ac3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:18.359781 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e696c7-6440-4a7f-b1f3-0c56fb1b3ac3 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:18.360061 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e696c7-6440-4a7f-b1f3-0c56fb1b3ac3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:18.360381 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e696c7-6440-4a7f-b1f3-0c56fb1b3ac3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:18.381994 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e696c7-6440-4a7f-b1f3-0c56fb1b3ac3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:18.383748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e696c7-6440-4a7f-b1f3-0c56fb1b3ac3 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:18.398244 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 923/1826] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:42:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:42:18.557097 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d0b88-3d89-44b6-9947-a9e7bebfaac2 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:18.557651 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d0b88-3d89-44b6-9947-a9e7bebfaac2 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:18.558070 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d0b88-3d89-44b6-9947-a9e7bebfaac2 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:18.559990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 904/1827] 10.222.0.33 () {54 vars in 921 bytes} [Mon Aug 8 08:42:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:42:18.584737 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03ecf4-1007-42a5-8107-f9d34cb1affd None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:18.585325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03ecf4-1007-42a5-8107-f9d34cb1affd None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:18.585732 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03ecf4-1007-42a5-8107-f9d34cb1affd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:18.624821 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0f03ecf4-1007-42a5-8107-f9d34cb1affd None None] MFA Rules not processed for user `a3317f751ec74bde95ec262394001292`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:18.626486 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0f03ecf4-1007-42a5-8107-f9d34cb1affd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:18.724583 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 924/1828] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Aug 8 08:42:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:19.080594 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d0b88-3d89-44b6-9947-a9e7bebfaac2 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:19.082326 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-bf4d0b88-3d89-44b6-9947-a9e7bebfaac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:19.126496 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d0b88-3d89-44b6-9947-a9e7bebfaac2 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:19.135111 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ec46e-4afa-4942-8a48-deba6783945f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:19.142778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ec46e-4afa-4942-8a48-deba6783945f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:19.142778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ec46e-4afa-4942-8a48-deba6783945f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:19.142778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ec46e-4afa-4942-8a48-deba6783945f admin admin] PATH_INFO: `/v3/projects/7f7642e3576241ef8903e4a4d9af1319` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:19.142778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274ec46e-4afa-4942-8a48-deba6783945f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f7642e3576241ef8903e4a4d9af1319)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:19.144616 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274ec46e-4afa-4942-8a48-deba6783945f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:19.242835 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03ecf4-1007-42a5-8107-f9d34cb1affd None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:19.244845 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0f03ecf4-1007-42a5-8107-f9d34cb1affd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:19.263242 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03ecf4-1007-42a5-8107-f9d34cb1affd None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:19.271308 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3813824-a7e8-47a6-925b-1921b56b0fa8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:19.271948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3813824-a7e8-47a6-925b-1921b56b0fa8 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:19.272223 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3813824-a7e8-47a6-925b-1921b56b0fa8 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:19.272496 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3813824-a7e8-47a6-925b-1921b56b0fa8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:19.282957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3813824-a7e8-47a6-925b-1921b56b0fa8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:19.284566 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3813824-a7e8-47a6-925b-1921b56b0fa8 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:19.288875 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:19.297846 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 925/1829] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:42:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:42:19.310493 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-274ec46e-4afa-4942-8a48-deba6783945f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f7642e3576241ef8903e4a4d9af1319', 'request_id': 'req-274ec46e-4afa-4942-8a48-deba6783945f'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:19.320145 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-274ec46e-4afa-4942-8a48-deba6783945f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f7642e3576241ef8903e4a4d9af1319', 'request_id': 'req-274ec46e-4afa-4942-8a48-deba6783945f'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:19.326341 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-274ec46e-4afa-4942-8a48-deba6783945f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f7642e3576241ef8903e4a4d9af1319', 'request_id': 'req-274ec46e-4afa-4942-8a48-deba6783945f'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:19.442910 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-274ec46e-4afa-4942-8a48-deba6783945f admin admin] The token cache is being invalidate because project 7f7642e3576241ef8903e4a4d9af1319 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:42:19.445593 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-274ec46e-4afa-4942-8a48-deba6783945f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:19.445593 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-274ec46e-4afa-4942-8a48-deba6783945f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:19.448110 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 905/1830] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:42:19 2022] DELETE /identity/v3/projects/7f7642e3576241ef8903e4a4d9af1319 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:19.452764 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3813824-a7e8-47a6-925b-1921b56b0fa8 service nova] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:19.466272 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3813824-a7e8-47a6-925b-1921b56b0fa8 service nova] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:19.543691 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8b4ad-9141-40e2-b3e9-b031eb780c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:19.544336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8b4ad-9141-40e2-b3e9-b031eb780c5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:19.544617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8b4ad-9141-40e2-b3e9-b031eb780c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:19.544900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8b4ad-9141-40e2-b3e9-b031eb780c5d admin admin] PATH_INFO: `/v3/users/218ca497ac5c4b3ba59dcfcceff0f7c5` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:19.546323 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d8b4ad-9141-40e2-b3e9-b031eb780c5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=218ca497ac5c4b3ba59dcfcceff0f7c5)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:19.548348 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d8b4ad-9141-40e2-b3e9-b031eb780c5d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:19.760567 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c7d8b4ad-9141-40e2-b3e9-b031eb780c5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '218ca497ac5c4b3ba59dcfcceff0f7c5'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:19.762384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c7d8b4ad-9141-40e2-b3e9-b031eb780c5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '218ca497ac5c4b3ba59dcfcceff0f7c5'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:19.763226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c7d8b4ad-9141-40e2-b3e9-b031eb780c5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '218ca497ac5c4b3ba59dcfcceff0f7c5'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:19.772703 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c7d8b4ad-9141-40e2-b3e9-b031eb780c5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '218ca497ac5c4b3ba59dcfcceff0f7c5'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:19.887414 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c7d8b4ad-9141-40e2-b3e9-b031eb780c5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '218ca497ac5c4b3ba59dcfcceff0f7c5'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:19.926777 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 926/1831] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:42:19 2022] DELETE /identity/v3/users/218ca497ac5c4b3ba59dcfcceff0f7c5 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:20.200708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ec46e-4afa-4942-8a48-deba6783945f admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:20.203569 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-274ec46e-4afa-4942-8a48-deba6783945f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:20.219270 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ec46e-4afa-4942-8a48-deba6783945f admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:20.306354 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f97a83-f28c-4592-9a81-cdbcdae1b149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:20.307042 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f97a83-f28c-4592-9a81-cdbcdae1b149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:20.307487 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f97a83-f28c-4592-9a81-cdbcdae1b149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:20.307928 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f97a83-f28c-4592-9a81-cdbcdae1b149 admin admin] PATH_INFO: `/v3/users/f395526c0c1847da8be96c7d0a8bcd58` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:20.310448 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f97a83-f28c-4592-9a81-cdbcdae1b149 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f395526c0c1847da8be96c7d0a8bcd58)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:20.313832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f97a83-f28c-4592-9a81-cdbcdae1b149 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:20.524919 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8b4ad-9141-40e2-b3e9-b031eb780c5d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:20.527415 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c7d8b4ad-9141-40e2-b3e9-b031eb780c5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:20.544612 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-b6f97a83-f28c-4592-9a81-cdbcdae1b149 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f395526c0c1847da8be96c7d0a8bcd58'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:20.546077 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-b6f97a83-f28c-4592-9a81-cdbcdae1b149 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f395526c0c1847da8be96c7d0a8bcd58'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:20.549053 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-b6f97a83-f28c-4592-9a81-cdbcdae1b149 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f395526c0c1847da8be96c7d0a8bcd58'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:20.556865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8b4ad-9141-40e2-b3e9-b031eb780c5d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:20.566602 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72b389-b967-4589-a536-3ced3cfa813a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:20.568075 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72b389-b967-4589-a536-3ced3cfa813a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:20.568432 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72b389-b967-4589-a536-3ced3cfa813a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:20.568776 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72b389-b967-4589-a536-3ced3cfa813a admin admin] PATH_INFO: `/v3/projects/5ab9bfbceeaa495c92f88a6d5a82a94c` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:20.570862 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f72b389-b967-4589-a536-3ced3cfa813a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ab9bfbceeaa495c92f88a6d5a82a94c)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:20.574433 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f72b389-b967-4589-a536-3ced3cfa813a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:20.576856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-b6f97a83-f28c-4592-9a81-cdbcdae1b149 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f395526c0c1847da8be96c7d0a8bcd58'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:20.612915 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:20.653231 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-b6f97a83-f28c-4592-9a81-cdbcdae1b149 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f395526c0c1847da8be96c7d0a8bcd58'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:20.674468 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 906/1832] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:42:20 2022] DELETE /identity/v3/users/f395526c0c1847da8be96c7d0a8bcd58 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:20.679125 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f013c-05dd-498e-9ed4-950973166304 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:20.679435 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f013c-05dd-498e-9ed4-950973166304 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:20.679706 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f013c-05dd-498e-9ed4-950973166304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:20.698830 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-4b6f013c-05dd-498e-9ed4-950973166304 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:20.700034 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-4b6f013c-05dd-498e-9ed4-950973166304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:20.759834 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9f72b389-b967-4589-a536-3ced3cfa813a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ab9bfbceeaa495c92f88a6d5a82a94c', 'request_id': 'req-9f72b389-b967-4589-a536-3ced3cfa813a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:20.813214 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 907/1833] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:20.821058 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f013c-05dd-498e-9ed4-950973166304 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:20.825954 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-4b6f013c-05dd-498e-9ed4-950973166304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:20.864047 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9f72b389-b967-4589-a536-3ced3cfa813a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ab9bfbceeaa495c92f88a6d5a82a94c', 'request_id': 'req-9f72b389-b967-4589-a536-3ced3cfa813a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:20.871529 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9f72b389-b967-4589-a536-3ced3cfa813a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ab9bfbceeaa495c92f88a6d5a82a94c', 'request_id': 'req-9f72b389-b967-4589-a536-3ced3cfa813a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:20.893664 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9f72b389-b967-4589-a536-3ced3cfa813a admin admin] The token cache is being invalidate because project 5ab9bfbceeaa495c92f88a6d5a82a94c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:42:20.893664 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9f72b389-b967-4589-a536-3ced3cfa813a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:20.895476 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9f72b389-b967-4589-a536-3ced3cfa813a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:20.899686 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 927/1834] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:42:20 2022] DELETE /identity/v3/projects/5ab9bfbceeaa495c92f88a6d5a82a94c => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:20.907358 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d1065-3ceb-43c0-8747-ee62b1e6eba2 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:20.907574 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d1065-3ceb-43c0-8747-ee62b1e6eba2 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:20.907856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d1065-3ceb-43c0-8747-ee62b1e6eba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:20.921080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f013c-05dd-498e-9ed4-950973166304 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:20.943826 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-884d1065-3ceb-43c0-8747-ee62b1e6eba2 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:20.945319 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-884d1065-3ceb-43c0-8747-ee62b1e6eba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:21.003042 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b0e7b-4536-467c-a417-1116fb974642 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:21.004630 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b0e7b-4536-467c-a417-1116fb974642 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:21.004976 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b0e7b-4536-467c-a417-1116fb974642 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:21.005379 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b0e7b-4536-467c-a417-1116fb974642 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:21.008174 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-029b0e7b-4536-467c-a417-1116fb974642 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:21.057579 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 928/1835] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:21.062241 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b87ea-b7ae-46ca-b841-6d8c83dce864 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:21.062706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b87ea-b7ae-46ca-b841-6d8c83dce864 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:21.063379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b87ea-b7ae-46ca-b841-6d8c83dce864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:21.088330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-f66b87ea-b7ae-46ca-b841-6d8c83dce864 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:21.089851 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f66b87ea-b7ae-46ca-b841-6d8c83dce864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:21.100006 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 929/1836] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:42:21.105219 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b87ea-b7ae-46ca-b841-6d8c83dce864 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:21.117502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b87ea-b7ae-46ca-b841-6d8c83dce864 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:21.118515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b0e7b-4536-467c-a417-1116fb974642 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:21.122587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b0e7b-4536-467c-a417-1116fb974642 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:21.122718 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d4d3b5-682c-427e-b7ce-9b507286f3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:21.123204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4d3b5-682c-427e-b7ce-9b507286f3fd admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:21.124013 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4d3b5-682c-427e-b7ce-9b507286f3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:21.124572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4d3b5-682c-427e-b7ce-9b507286f3fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:21.125687 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d4d3b5-682c-427e-b7ce-9b507286f3fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:42:21.126147 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d4d3b5-682c-427e-b7ce-9b507286f3fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:21.127753 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d4d3b5-682c-427e-b7ce-9b507286f3fd admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:21.132465 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 908/1837] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:42:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:42:21.143132 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 930/1838] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:42:21.148219 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b0e7b-4536-467c-a417-1116fb974642 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:21.158728 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b0e7b-4536-467c-a417-1116fb974642 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:21.164422 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a466c-2b81-4c55-b79f-d7ac5f5e016d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:21.165176 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a466c-2b81-4c55-b79f-d7ac5f5e016d admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:21.165536 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a466c-2b81-4c55-b79f-d7ac5f5e016d admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:21.165930 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a466c-2b81-4c55-b79f-d7ac5f5e016d admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:21.170851 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63a466c-2b81-4c55-b79f-d7ac5f5e016d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:21.170851 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63a466c-2b81-4c55-b79f-d7ac5f5e016d admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:21.259674 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 909/1839] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:42:21 2022] POST /identity/v3/projects => generated 369 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:42:21.260902 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d4d3b5-682c-427e-b7ce-9b507286f3fd admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:21.271522 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d4d3b5-682c-427e-b7ce-9b507286f3fd admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:21.331633 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a87b3-3720-4424-9970-7ac9157d8786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:21.332245 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a87b3-3720-4424-9970-7ac9157d8786 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:21.332518 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a87b3-3720-4424-9970-7ac9157d8786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:21.332835 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a87b3-3720-4424-9970-7ac9157d8786 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:21.333513 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204a87b3-3720-4424-9970-7ac9157d8786 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:21.335704 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204a87b3-3720-4424-9970-7ac9157d8786 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:21.428844 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 931/1840] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:42:21 2022] POST /identity/v3/users => generated 285 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:42:21.432911 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a466c-2b81-4c55-b79f-d7ac5f5e016d admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:21.470206 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a466c-2b81-4c55-b79f-d7ac5f5e016d admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:21.489997 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5988304-1573-4e98-9686-3fba2900be25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:21.490629 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5988304-1573-4e98-9686-3fba2900be25 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:21.491081 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5988304-1573-4e98-9686-3fba2900be25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:21.491352 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5988304-1573-4e98-9686-3fba2900be25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:21.492110 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5988304-1573-4e98-9686-3fba2900be25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:21.495008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5988304-1573-4e98-9686-3fba2900be25 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:21.521311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 910/1841] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:42:21.525296 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a87b3-3720-4424-9970-7ac9157d8786 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:21.536541 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a87b3-3720-4424-9970-7ac9157d8786 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:21.541784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50bc97d-8f08-4c6e-a4c2-d91bf4db3aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:21.542568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bc97d-8f08-4c6e-a4c2-d91bf4db3aa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:21.542841 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bc97d-8f08-4c6e-a4c2-d91bf4db3aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:21.543372 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bc97d-8f08-4c6e-a4c2-d91bf4db3aa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:21.544084 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50bc97d-8f08-4c6e-a4c2-d91bf4db3aa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:21.544704 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:21.546708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50bc97d-8f08-4c6e-a4c2-d91bf4db3aa5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:21.559013 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 932/1842] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:42:21.564157 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5988304-1573-4e98-9686-3fba2900be25 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:21.575040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5988304-1573-4e98-9686-3fba2900be25 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:21.580163 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69367786-4250-4c6f-8418-762b8ab1e55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:21.580851 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69367786-4250-4c6f-8418-762b8ab1e55d admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:21.581130 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69367786-4250-4c6f-8418-762b8ab1e55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:21.584530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69367786-4250-4c6f-8418-762b8ab1e55d admin admin] PATH_INFO: `/v3/projects/968732d328d94879a5edb49a64ab3670/users/5bc5329934e2438fb9adbe5aff0fb14b/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:21.599940 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69367786-4250-4c6f-8418-762b8ab1e55d admin admin] RBAC: Authorizing `identity:create_grant(project_id=968732d328d94879a5edb49a64ab3670, user_id=5bc5329934e2438fb9adbe5aff0fb14b, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:21.602808 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69367786-4250-4c6f-8418-762b8ab1e55d admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:21.722085 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 911/1843] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:42:21 2022] PUT /identity/v3/projects/968732d328d94879a5edb49a64ab3670/users/5bc5329934e2438fb9adbe5aff0fb14b/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:21.733842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85839467-aab8-47f6-9a42-e17ff5ecdc00 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:21.734211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85839467-aab8-47f6-9a42-e17ff5ecdc00 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:21.734549 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85839467-aab8-47f6-9a42-e17ff5ecdc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:21.767918 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-85839467-aab8-47f6-9a42-e17ff5ecdc00 None None] MFA Rules not processed for user `5bc5329934e2438fb9adbe5aff0fb14b`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:21.769312 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-85839467-aab8-47f6-9a42-e17ff5ecdc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:21.855386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 933/1844] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:21.868675 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69367786-4250-4c6f-8418-762b8ab1e55d admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:21.870589 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-69367786-4250-4c6f-8418-762b8ab1e55d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:21.915813 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85839467-aab8-47f6-9a42-e17ff5ecdc00 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:21.917281 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-85839467-aab8-47f6-9a42-e17ff5ecdc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:21.994961 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69367786-4250-4c6f-8418-762b8ab1e55d admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:21.999020 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad380b2-5ac9-4bba-8dfa-e9b4fba10b13 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '627171bf2a2e4a21ba6f800879f8c665', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:22.000004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad380b2-5ac9-4bba-8dfa-e9b4fba10b13 service glance] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:22.003496 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad380b2-5ac9-4bba-8dfa-e9b4fba10b13 service glance] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:22.003496 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad380b2-5ac9-4bba-8dfa-e9b4fba10b13 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:22.009327 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85839467-aab8-47f6-9a42-e17ff5ecdc00 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:22.010813 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad380b2-5ac9-4bba-8dfa-e9b4fba10b13 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:22.012454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad380b2-5ac9-4bba-8dfa-e9b4fba10b13 service glance] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:22.016254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa487268-308e-4299-a675-38aca2df9c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:22.016962 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa487268-308e-4299-a675-38aca2df9c8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:22.017454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa487268-308e-4299-a675-38aca2df9c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:22.020275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa487268-308e-4299-a675-38aca2df9c8c admin admin] PATH_INFO: `/v3/projects/ca71fb3a1e294249acf42196e205a258` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:22.020275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa487268-308e-4299-a675-38aca2df9c8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca71fb3a1e294249acf42196e205a258)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:22.021366 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa487268-308e-4299-a675-38aca2df9c8c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:22.024882 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 912/1845] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:42:21 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:42:22.158984 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-aa487268-308e-4299-a675-38aca2df9c8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca71fb3a1e294249acf42196e205a258', 'request_id': 'req-aa487268-308e-4299-a675-38aca2df9c8c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:22.259027 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-aa487268-308e-4299-a675-38aca2df9c8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca71fb3a1e294249acf42196e205a258', 'request_id': 'req-aa487268-308e-4299-a675-38aca2df9c8c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:22.268819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-aa487268-308e-4299-a675-38aca2df9c8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca71fb3a1e294249acf42196e205a258', 'request_id': 'req-aa487268-308e-4299-a675-38aca2df9c8c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:22.298089 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-aa487268-308e-4299-a675-38aca2df9c8c admin admin] The token cache is being invalidate because project ca71fb3a1e294249acf42196e205a258 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:42:22.298385 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-aa487268-308e-4299-a675-38aca2df9c8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:22.299069 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-aa487268-308e-4299-a675-38aca2df9c8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:22.300699 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 934/1846] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:42:21 2022] DELETE /identity/v3/projects/ca71fb3a1e294249acf42196e205a258 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:42:22.305247 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad380b2-5ac9-4bba-8dfa-e9b4fba10b13 service glance] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:22.323911 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad380b2-5ac9-4bba-8dfa-e9b4fba10b13 service glance] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:22.425376 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc7b2c-cea8-4bd8-bd04-f16fc5c2ef40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:22.426035 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc7b2c-cea8-4bd8-bd04-f16fc5c2ef40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:22.426317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc7b2c-cea8-4bd8-bd04-f16fc5c2ef40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:22.426589 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc7b2c-cea8-4bd8-bd04-f16fc5c2ef40 admin admin] PATH_INFO: `/v3/users/22b810dbed6144c89dbcc3cee9e725a4` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:22.429899 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc7b2c-cea8-4bd8-bd04-f16fc5c2ef40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22b810dbed6144c89dbcc3cee9e725a4)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:22.431871 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc7b2c-cea8-4bd8-bd04-f16fc5c2ef40 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:22.459899 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:22.689424 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-b1fc7b2c-cea8-4bd8-bd04-f16fc5c2ef40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22b810dbed6144c89dbcc3cee9e725a4'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:22.690333 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-b1fc7b2c-cea8-4bd8-bd04-f16fc5c2ef40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22b810dbed6144c89dbcc3cee9e725a4'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:22.691038 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-b1fc7b2c-cea8-4bd8-bd04-f16fc5c2ef40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22b810dbed6144c89dbcc3cee9e725a4'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:22.698170 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-b1fc7b2c-cea8-4bd8-bd04-f16fc5c2ef40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22b810dbed6144c89dbcc3cee9e725a4'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:22.784110 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-b1fc7b2c-cea8-4bd8-bd04-f16fc5c2ef40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22b810dbed6144c89dbcc3cee9e725a4'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:22.802369 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 913/1847] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:42:22 2022] DELETE /identity/v3/users/22b810dbed6144c89dbcc3cee9e725a4 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:23.365316 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa487268-308e-4299-a675-38aca2df9c8c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:23.366925 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-aa487268-308e-4299-a675-38aca2df9c8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:23.409940 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa487268-308e-4299-a675-38aca2df9c8c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:23.497995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb62f23-3100-43c1-8a67-25c200f71af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:23.499366 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb62f23-3100-43c1-8a67-25c200f71af0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:23.499712 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb62f23-3100-43c1-8a67-25c200f71af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:23.499996 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb62f23-3100-43c1-8a67-25c200f71af0 admin admin] PATH_INFO: `/v3/projects/e862bba3857c4a30b381968d32d02bab` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:23.501212 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb62f23-3100-43c1-8a67-25c200f71af0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e862bba3857c4a30b381968d32d02bab)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:23.503460 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb62f23-3100-43c1-8a67-25c200f71af0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:23.527622 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:23.637603 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7bb62f23-3100-43c1-8a67-25c200f71af0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e862bba3857c4a30b381968d32d02bab', 'request_id': 'req-7bb62f23-3100-43c1-8a67-25c200f71af0'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:23.744430 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7bb62f23-3100-43c1-8a67-25c200f71af0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e862bba3857c4a30b381968d32d02bab', 'request_id': 'req-7bb62f23-3100-43c1-8a67-25c200f71af0'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:23.752109 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7bb62f23-3100-43c1-8a67-25c200f71af0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e862bba3857c4a30b381968d32d02bab', 'request_id': 'req-7bb62f23-3100-43c1-8a67-25c200f71af0'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:23.763468 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7bb62f23-3100-43c1-8a67-25c200f71af0 admin admin] The token cache is being invalidate because project e862bba3857c4a30b381968d32d02bab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:42:23.763762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7bb62f23-3100-43c1-8a67-25c200f71af0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:23.764458 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7bb62f23-3100-43c1-8a67-25c200f71af0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:23.767919 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 935/1848] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:42:23 2022] DELETE /identity/v3/projects/e862bba3857c4a30b381968d32d02bab => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:23.773099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ebcf5-d157-4b3d-a434-5981132a0e58 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:23.773489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ebcf5-d157-4b3d-a434-5981132a0e58 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:23.773818 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ebcf5-d157-4b3d-a434-5981132a0e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:23.808059 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-da0ebcf5-d157-4b3d-a434-5981132a0e58 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:23.809532 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-da0ebcf5-d157-4b3d-a434-5981132a0e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:23.885205 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 914/1849] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:23.887158 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bce51-5373-42bc-8e1d-c675d96b3e60 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:23.887441 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bce51-5373-42bc-8e1d-c675d96b3e60 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:23.887742 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bce51-5373-42bc-8e1d-c675d96b3e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:23.909716 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-323bce51-5373-42bc-8e1d-c675d96b3e60 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:23.911713 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-323bce51-5373-42bc-8e1d-c675d96b3e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:23.921202 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 936/1850] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:42:23.925989 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ebcf5-d157-4b3d-a434-5981132a0e58 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:23.938026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ebcf5-d157-4b3d-a434-5981132a0e58 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:23.943011 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb2411-1963-4811-80fa-29acaeed87ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:23.945785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb2411-1963-4811-80fa-29acaeed87ab admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:23.945785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb2411-1963-4811-80fa-29acaeed87ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:23.945785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb2411-1963-4811-80fa-29acaeed87ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:23.945785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb2411-1963-4811-80fa-29acaeed87ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:42:23.945785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb2411-1963-4811-80fa-29acaeed87ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:23.946642 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb2411-1963-4811-80fa-29acaeed87ab admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:23.958968 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 915/1851] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:42:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:42:23.963579 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323bce51-5373-42bc-8e1d-c675d96b3e60 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:23.973871 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323bce51-5373-42bc-8e1d-c675d96b3e60 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:23.979356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b5885-e7b4-4794-8543-833ccc1afa6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:23.979995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b5885-e7b4-4794-8543-833ccc1afa6d admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:23.980275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b5885-e7b4-4794-8543-833ccc1afa6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:23.980547 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b5885-e7b4-4794-8543-833ccc1afa6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:23.981229 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564b5885-e7b4-4794-8543-833ccc1afa6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:23.983628 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564b5885-e7b4-4794-8543-833ccc1afa6d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:24.183603 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 937/1852] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:42:23 2022] POST /identity/v3/projects => generated 381 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:42:24.188191 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb2411-1963-4811-80fa-29acaeed87ab admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:24.210831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb2411-1963-4811-80fa-29acaeed87ab admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:24.347542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee64bd9-2f21-4247-8747-9e3d55372a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:24.348313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee64bd9-2f21-4247-8747-9e3d55372a53 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:24.348706 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee64bd9-2f21-4247-8747-9e3d55372a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:24.348982 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee64bd9-2f21-4247-8747-9e3d55372a53 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:24.349755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee64bd9-2f21-4247-8747-9e3d55372a53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:24.351689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee64bd9-2f21-4247-8747-9e3d55372a53 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:24.492827 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 916/1853] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:42:24 2022] POST /identity/v3/users => generated 291 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:42:24.497029 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b5885-e7b4-4794-8543-833ccc1afa6d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:24.508611 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b5885-e7b4-4794-8543-833ccc1afa6d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:24.513763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a923318-9325-4865-86b0-25987b9540a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:24.529511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a923318-9325-4865-86b0-25987b9540a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:24.529511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a923318-9325-4865-86b0-25987b9540a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:24.529511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a923318-9325-4865-86b0-25987b9540a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:24.529511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a923318-9325-4865-86b0-25987b9540a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:24.531693 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a923318-9325-4865-86b0-25987b9540a4 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:24.555691 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 938/1854] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:42:24.559836 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee64bd9-2f21-4247-8747-9e3d55372a53 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:24.572093 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee64bd9-2f21-4247-8747-9e3d55372a53 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:24.577971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e6697-9402-4bcf-9428-a3fb3bc7ead3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:24.578674 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e6697-9402-4bcf-9428-a3fb3bc7ead3 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:24.578962 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e6697-9402-4bcf-9428-a3fb3bc7ead3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:24.579316 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e6697-9402-4bcf-9428-a3fb3bc7ead3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:24.580042 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292e6697-9402-4bcf-9428-a3fb3bc7ead3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:24.581781 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292e6697-9402-4bcf-9428-a3fb3bc7ead3 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:24.597239 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 917/1855] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:42:24.598368 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a923318-9325-4865-86b0-25987b9540a4 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:24.613940 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a923318-9325-4865-86b0-25987b9540a4 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:24.619786 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb907f-c052-4eb8-902c-5bd446262722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:24.620465 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb907f-c052-4eb8-902c-5bd446262722 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:24.620894 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb907f-c052-4eb8-902c-5bd446262722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:24.621336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb907f-c052-4eb8-902c-5bd446262722 admin admin] PATH_INFO: `/v3/projects/d16bc92917f4409480c3d99b6d8654b7/users/54b41b3033b9433e957f112b5cd4c314/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:24.672149 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb907f-c052-4eb8-902c-5bd446262722 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d16bc92917f4409480c3d99b6d8654b7, user_id=54b41b3033b9433e957f112b5cd4c314, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:24.676972 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb907f-c052-4eb8-902c-5bd446262722 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:24.836065 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:24.857591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 939/1856] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:42:24 2022] PUT /identity/v3/projects/d16bc92917f4409480c3d99b6d8654b7/users/54b41b3033b9433e957f112b5cd4c314/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:42:24.860474 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54798e1b-3460-4130-8da3-699309a44a56 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:24.860761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54798e1b-3460-4130-8da3-699309a44a56 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:24.861185 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54798e1b-3460-4130-8da3-699309a44a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:24.894209 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:24.938050 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-54798e1b-3460-4130-8da3-699309a44a56 None None] MFA Rules not processed for user `54b41b3033b9433e957f112b5cd4c314`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:24.939391 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-54798e1b-3460-4130-8da3-699309a44a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:25.025613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 918/1857] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:25.030740 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb907f-c052-4eb8-902c-5bd446262722 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:25.102290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb907f-c052-4eb8-902c-5bd446262722 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:25.109874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd1dbd-ef17-4514-ae4e-e7eade5f15d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:25.110502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd1dbd-ef17-4514-ae4e-e7eade5f15d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:25.110785 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd1dbd-ef17-4514-ae4e-e7eade5f15d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:25.111063 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd1dbd-ef17-4514-ae4e-e7eade5f15d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:25.111711 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd1dbd-ef17-4514-ae4e-e7eade5f15d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:25.113835 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd1dbd-ef17-4514-ae4e-e7eade5f15d8 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:25.227930 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 940/1858] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:42:25 2022] POST /identity/v3/projects => generated 379 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:42:25.231759 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54798e1b-3460-4130-8da3-699309a44a56 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:25.252633 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54798e1b-3460-4130-8da3-699309a44a56 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:25.321176 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8e5726-58f7-4f66-89a0-ba4adfbb4e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:25.321941 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e5726-58f7-4f66-89a0-ba4adfbb4e54 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:25.322294 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e5726-58f7-4f66-89a0-ba4adfbb4e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:25.322626 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e5726-58f7-4f66-89a0-ba4adfbb4e54 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:25.323318 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8e5726-58f7-4f66-89a0-ba4adfbb4e54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:25.325380 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8e5726-58f7-4f66-89a0-ba4adfbb4e54 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:25.402151 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 919/1859] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:42:25 2022] POST /identity/v3/users => generated 296 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:42:25.406635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd1dbd-ef17-4514-ae4e-e7eade5f15d8 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:25.422714 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd1dbd-ef17-4514-ae4e-e7eade5f15d8 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:25.428184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bb8eb3-e7df-47f6-afe6-cc7f6848c170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:25.428887 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb8eb3-e7df-47f6-afe6-cc7f6848c170 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:25.429281 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb8eb3-e7df-47f6-afe6-cc7f6848c170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:25.429622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb8eb3-e7df-47f6-afe6-cc7f6848c170 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:25.430339 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bb8eb3-e7df-47f6-afe6-cc7f6848c170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:25.432010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bb8eb3-e7df-47f6-afe6-cc7f6848c170 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:25.447721 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 941/1860] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:42:25.454860 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8e5726-58f7-4f66-89a0-ba4adfbb4e54 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:25.471483 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8e5726-58f7-4f66-89a0-ba4adfbb4e54 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:25.478699 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c82432-6d6d-4f88-a15f-46d9ab2b6158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:25.479653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c82432-6d6d-4f88-a15f-46d9ab2b6158 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:25.480143 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c82432-6d6d-4f88-a15f-46d9ab2b6158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:25.480552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c82432-6d6d-4f88-a15f-46d9ab2b6158 admin admin] PATH_INFO: `/v3/projects/1d70fc7f673b49868425cc9d618ba623/users/e74f24dcc9734815bcebd1d45ccfd615/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:25.504774 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c82432-6d6d-4f88-a15f-46d9ab2b6158 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d70fc7f673b49868425cc9d618ba623, user_id=e74f24dcc9734815bcebd1d45ccfd615, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:25.507541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c82432-6d6d-4f88-a15f-46d9ab2b6158 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:25.597340 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 920/1861] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:42:25 2022] PUT /identity/v3/projects/1d70fc7f673b49868425cc9d618ba623/users/e74f24dcc9734815bcebd1d45ccfd615/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:42:25.601599 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bb8eb3-e7df-47f6-afe6-cc7f6848c170 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:25.617775 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bb8eb3-e7df-47f6-afe6-cc7f6848c170 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:25.697690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14e9eb-b184-4eca-a6d9-c7f1f0f68cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:25.698360 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14e9eb-b184-4eca-a6d9-c7f1f0f68cca admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:25.698756 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14e9eb-b184-4eca-a6d9-c7f1f0f68cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:25.699130 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14e9eb-b184-4eca-a6d9-c7f1f0f68cca admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:25.699874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da14e9eb-b184-4eca-a6d9-c7f1f0f68cca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:25.702162 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da14e9eb-b184-4eca-a6d9-c7f1f0f68cca admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:25.713748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 942/1862] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:42:25.718271 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c82432-6d6d-4f88-a15f-46d9ab2b6158 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:25.741423 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c82432-6d6d-4f88-a15f-46d9ab2b6158 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:25.755346 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af77239a-ea3b-4396-a34e-0e8bc1cd9416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:25.755346 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77239a-ea3b-4396-a34e-0e8bc1cd9416 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:25.755346 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77239a-ea3b-4396-a34e-0e8bc1cd9416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:25.755346 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77239a-ea3b-4396-a34e-0e8bc1cd9416 admin admin] PATH_INFO: `/v3/projects/1d70fc7f673b49868425cc9d618ba623/users/e74f24dcc9734815bcebd1d45ccfd615/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:25.767007 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af77239a-ea3b-4396-a34e-0e8bc1cd9416 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d70fc7f673b49868425cc9d618ba623, user_id=e74f24dcc9734815bcebd1d45ccfd615, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:25.768700 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af77239a-ea3b-4396-a34e-0e8bc1cd9416 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:25.789197 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 921/1863] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:42:25 2022] PUT /identity/v3/projects/1d70fc7f673b49868425cc9d618ba623/users/e74f24dcc9734815bcebd1d45ccfd615/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:25.793498 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b7908-c09f-4271-a95c-43f2993969af None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:25.793956 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b7908-c09f-4271-a95c-43f2993969af None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:25.794519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b7908-c09f-4271-a95c-43f2993969af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:25.837529 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-7f8b7908-c09f-4271-a95c-43f2993969af None None] MFA Rules not processed for user `e74f24dcc9734815bcebd1d45ccfd615`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:25.848654 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7f8b7908-c09f-4271-a95c-43f2993969af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:25.959385 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 943/1864] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:25 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:25.976075 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af77239a-ea3b-4396-a34e-0e8bc1cd9416 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:25.978201 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-af77239a-ea3b-4396-a34e-0e8bc1cd9416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:26.057254 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af77239a-ea3b-4396-a34e-0e8bc1cd9416 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:26.062542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e501c-ba3b-4674-8cfc-2afcaa9dea42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:26.063459 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e501c-ba3b-4674-8cfc-2afcaa9dea42 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:26.063956 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e501c-ba3b-4674-8cfc-2afcaa9dea42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:26.064392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e501c-ba3b-4674-8cfc-2afcaa9dea42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:26.075072 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e501c-ba3b-4674-8cfc-2afcaa9dea42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:26.077956 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e501c-ba3b-4674-8cfc-2afcaa9dea42 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:26.093259 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 922/1865] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:42:25 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:42:26.269855 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:26.356595 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:27.449669 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b7908-c09f-4271-a95c-43f2993969af None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:27.460933 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b7908-c09f-4271-a95c-43f2993969af None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:27.483310 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd3959-8a9f-4353-9042-1d1ae916e2aa tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e74f24dcc9734815bcebd1d45ccfd615', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d70fc7f673b49868425cc9d618ba623', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:27.484357 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd3959-8a9f-4353-9042-1d1ae916e2aa tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:27.484730 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd3959-8a9f-4353-9042-1d1ae916e2aa tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:27.485122 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd3959-8a9f-4353-9042-1d1ae916e2aa tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:27.486081 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd3959-8a9f-4353-9042-1d1ae916e2aa tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:42:27.487011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd3959-8a9f-4353-9042-1d1ae916e2aa tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:27.491307 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd3959-8a9f-4353-9042-1d1ae916e2aa tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:27.518080 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 944/1866] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:42:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:42:27.528854 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e501c-ba3b-4674-8cfc-2afcaa9dea42 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:27.540776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e501c-ba3b-4674-8cfc-2afcaa9dea42 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:27.547068 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba53e1-4c58-4814-965c-a54e16c66b37 tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e74f24dcc9734815bcebd1d45ccfd615', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d70fc7f673b49868425cc9d618ba623', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:27.549139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba53e1-4c58-4814-965c-a54e16c66b37 tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:27.549139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba53e1-4c58-4814-965c-a54e16c66b37 tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:27.549139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba53e1-4c58-4814-965c-a54e16c66b37 tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:27.549422 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba53e1-4c58-4814-965c-a54e16c66b37 tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:27.551528 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba53e1-4c58-4814-965c-a54e16c66b37 tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:27.689640 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 923/1867] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:42:27 2022] POST /identity/v3/projects => generated 354 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:42:27.693951 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd3959-8a9f-4353-9042-1d1ae916e2aa tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:27.710052 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd3959-8a9f-4353-9042-1d1ae916e2aa tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:27.795880 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf38597-2f87-42bb-92ba-8600df7ea54e tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e74f24dcc9734815bcebd1d45ccfd615', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d70fc7f673b49868425cc9d618ba623', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:27.795880 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf38597-2f87-42bb-92ba-8600df7ea54e tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:27.795880 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf38597-2f87-42bb-92ba-8600df7ea54e tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:27.796172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf38597-2f87-42bb-92ba-8600df7ea54e tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:27.796172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf38597-2f87-42bb-92ba-8600df7ea54e tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:42:27.796172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf38597-2f87-42bb-92ba-8600df7ea54e tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:27.797866 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf38597-2f87-42bb-92ba-8600df7ea54e tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:27.812010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 945/1868] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:42:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:42:27.841106 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:27.866186 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba53e1-4c58-4814-965c-a54e16c66b37 tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:27.881546 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba53e1-4c58-4814-965c-a54e16c66b37 tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:27.887627 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54beee01-62ab-46f9-ab58-d127c06aa03a tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e74f24dcc9734815bcebd1d45ccfd615', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d70fc7f673b49868425cc9d618ba623', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:27.888385 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54beee01-62ab-46f9-ab58-d127c06aa03a tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:27.888810 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54beee01-62ab-46f9-ab58-d127c06aa03a tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:27.889276 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54beee01-62ab-46f9-ab58-d127c06aa03a tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] PATH_INFO: `/v3/projects/20dc675571ba424e8f2f543010d38b19` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:27.890594 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54beee01-62ab-46f9-ab58-d127c06aa03a tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] RBAC: Authorizing `identity:delete_project(project_id=20dc675571ba424e8f2f543010d38b19)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:27.892630 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54beee01-62ab-46f9-ab58-d127c06aa03a tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:27.954915 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:28.070229 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-54beee01-62ab-46f9-ab58-d127c06aa03a tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20dc675571ba424e8f2f543010d38b19', 'request_id': 'req-54beee01-62ab-46f9-ab58-d127c06aa03a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:28.078316 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-54beee01-62ab-46f9-ab58-d127c06aa03a tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20dc675571ba424e8f2f543010d38b19', 'request_id': 'req-54beee01-62ab-46f9-ab58-d127c06aa03a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:28.089736 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-54beee01-62ab-46f9-ab58-d127c06aa03a tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20dc675571ba424e8f2f543010d38b19', 'request_id': 'req-54beee01-62ab-46f9-ab58-d127c06aa03a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:28.171741 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-54beee01-62ab-46f9-ab58-d127c06aa03a tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] The token cache is being invalidate because project 20dc675571ba424e8f2f543010d38b19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:42:28.173024 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-54beee01-62ab-46f9-ab58-d127c06aa03a tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:28.173024 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-54beee01-62ab-46f9-ab58-d127c06aa03a tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:28.174691 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 924/1869] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:42:27 2022] DELETE /identity/v3/projects/20dc675571ba424e8f2f543010d38b19 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:29.792511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf38597-2f87-42bb-92ba-8600df7ea54e tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:29.794599 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0bf38597-2f87-42bb-92ba-8600df7ea54e tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:29.891564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf38597-2f87-42bb-92ba-8600df7ea54e tempest-VolumeQuotaClassesTest-730145546 tempest-VolumeQuotaClassesTest-730145546-project-admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:29.954833 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057bb54e-336f-4e79-bd0e-058b73dd3a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:29.955621 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057bb54e-336f-4e79-bd0e-058b73dd3a08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:29.955899 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057bb54e-336f-4e79-bd0e-058b73dd3a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:29.956255 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057bb54e-336f-4e79-bd0e-058b73dd3a08 admin admin] PATH_INFO: `/v3/users/54b41b3033b9433e957f112b5cd4c314` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:29.957383 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057bb54e-336f-4e79-bd0e-058b73dd3a08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54b41b3033b9433e957f112b5cd4c314)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:29.959431 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057bb54e-336f-4e79-bd0e-058b73dd3a08 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:30.113244 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:30.121760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-057bb54e-336f-4e79-bd0e-058b73dd3a08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54b41b3033b9433e957f112b5cd4c314'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:30.122411 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-057bb54e-336f-4e79-bd0e-058b73dd3a08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54b41b3033b9433e957f112b5cd4c314'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:30.123023 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-057bb54e-336f-4e79-bd0e-058b73dd3a08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54b41b3033b9433e957f112b5cd4c314'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:30.133028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-057bb54e-336f-4e79-bd0e-058b73dd3a08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54b41b3033b9433e957f112b5cd4c314'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:30.201572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-057bb54e-336f-4e79-bd0e-058b73dd3a08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54b41b3033b9433e957f112b5cd4c314'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:30.222062 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 946/1870] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:42:29 2022] DELETE /identity/v3/users/54b41b3033b9433e957f112b5cd4c314 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:30.227350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc8fc8-a8c3-453c-9c05-570ba708ac87 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:30.227895 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc8fc8-a8c3-453c-9c05-570ba708ac87 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:30.227982 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc8fc8-a8c3-453c-9c05-570ba708ac87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:30.246987 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-7ecc8fc8-a8c3-453c-9c05-570ba708ac87 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:30.248928 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-7ecc8fc8-a8c3-453c-9c05-570ba708ac87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:30.308298 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 925/1871] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:42:30.320137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057bb54e-336f-4e79-bd0e-058b73dd3a08 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:30.321764 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-057bb54e-336f-4e79-bd0e-058b73dd3a08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:30.381912 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057bb54e-336f-4e79-bd0e-058b73dd3a08 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:30.385419 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5573d8e-2ec5-49b2-a6c0-5e57ded4da61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:30.386026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5573d8e-2ec5-49b2-a6c0-5e57ded4da61 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:30.386301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5573d8e-2ec5-49b2-a6c0-5e57ded4da61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:30.386573 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5573d8e-2ec5-49b2-a6c0-5e57ded4da61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:30.396949 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5573d8e-2ec5-49b2-a6c0-5e57ded4da61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:30.398464 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5573d8e-2ec5-49b2-a6c0-5e57ded4da61 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:30.414849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 947/1872] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:42:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:42:30.996216 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc8fc8-a8c3-453c-9c05-570ba708ac87 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:30.998709 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-7ecc8fc8-a8c3-453c-9c05-570ba708ac87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:31.017075 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc8fc8-a8c3-453c-9c05-570ba708ac87 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:31.025648 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c2b7e-294c-4e15-86ab-a6b1b619bc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:31.026391 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c2b7e-294c-4e15-86ab-a6b1b619bc78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:31.028954 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c2b7e-294c-4e15-86ab-a6b1b619bc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:31.030035 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c2b7e-294c-4e15-86ab-a6b1b619bc78 admin admin] PATH_INFO: `/v3/projects/d16bc92917f4409480c3d99b6d8654b7` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:31.032027 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c2b7e-294c-4e15-86ab-a6b1b619bc78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d16bc92917f4409480c3d99b6d8654b7)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:31.040490 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c2b7e-294c-4e15-86ab-a6b1b619bc78 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:31.188229 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-638c2b7e-294c-4e15-86ab-a6b1b619bc78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd16bc92917f4409480c3d99b6d8654b7', 'request_id': 'req-638c2b7e-294c-4e15-86ab-a6b1b619bc78'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:31.201183 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-638c2b7e-294c-4e15-86ab-a6b1b619bc78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd16bc92917f4409480c3d99b6d8654b7', 'request_id': 'req-638c2b7e-294c-4e15-86ab-a6b1b619bc78'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:31.214595 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-638c2b7e-294c-4e15-86ab-a6b1b619bc78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd16bc92917f4409480c3d99b6d8654b7', 'request_id': 'req-638c2b7e-294c-4e15-86ab-a6b1b619bc78'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:31.298127 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-638c2b7e-294c-4e15-86ab-a6b1b619bc78 admin admin] The token cache is being invalidate because project d16bc92917f4409480c3d99b6d8654b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:42:31.299189 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-638c2b7e-294c-4e15-86ab-a6b1b619bc78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:31.300033 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-638c2b7e-294c-4e15-86ab-a6b1b619bc78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:31.301844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 926/1873] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:42:30 2022] DELETE /identity/v3/projects/d16bc92917f4409480c3d99b6d8654b7 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:31.306084 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5573d8e-2ec5-49b2-a6c0-5e57ded4da61 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:31.317378 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5573d8e-2ec5-49b2-a6c0-5e57ded4da61 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:31.382613 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff7c07-04e3-4a08-a732-10f16c15fb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:31.383264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff7c07-04e3-4a08-a732-10f16c15fb18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:31.383542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff7c07-04e3-4a08-a732-10f16c15fb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:31.383815 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff7c07-04e3-4a08-a732-10f16c15fb18 admin admin] PATH_INFO: `/v3/users/e74f24dcc9734815bcebd1d45ccfd615` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:31.385014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff7c07-04e3-4a08-a732-10f16c15fb18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e74f24dcc9734815bcebd1d45ccfd615)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:31.391679 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff7c07-04e3-4a08-a732-10f16c15fb18 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:31.604658 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-deff7c07-04e3-4a08-a732-10f16c15fb18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e74f24dcc9734815bcebd1d45ccfd615'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:31.605557 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-deff7c07-04e3-4a08-a732-10f16c15fb18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e74f24dcc9734815bcebd1d45ccfd615'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:31.606270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-deff7c07-04e3-4a08-a732-10f16c15fb18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e74f24dcc9734815bcebd1d45ccfd615'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:31.615132 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-deff7c07-04e3-4a08-a732-10f16c15fb18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e74f24dcc9734815bcebd1d45ccfd615'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:31.625591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:31.691739 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-deff7c07-04e3-4a08-a732-10f16c15fb18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e74f24dcc9734815bcebd1d45ccfd615'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:31.718378 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 948/1874] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:42:31 2022] DELETE /identity/v3/users/e74f24dcc9734815bcebd1d45ccfd615 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:32.221974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c2b7e-294c-4e15-86ab-a6b1b619bc78 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:32.223810 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-638c2b7e-294c-4e15-86ab-a6b1b619bc78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:32.239758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c2b7e-294c-4e15-86ab-a6b1b619bc78 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:32.269521 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:32.324582 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1eda83-2bac-406b-8e58-e1ccd2b52f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:32.325258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eda83-2bac-406b-8e58-e1ccd2b52f5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:32.325622 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eda83-2bac-406b-8e58-e1ccd2b52f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:32.325890 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eda83-2bac-406b-8e58-e1ccd2b52f5b admin admin] PATH_INFO: `/v3/projects/1d70fc7f673b49868425cc9d618ba623` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:32.327145 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1eda83-2bac-406b-8e58-e1ccd2b52f5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d70fc7f673b49868425cc9d618ba623)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:32.329525 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1eda83-2bac-406b-8e58-e1ccd2b52f5b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:32.444065 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-9e1eda83-2bac-406b-8e58-e1ccd2b52f5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d70fc7f673b49868425cc9d618ba623', 'request_id': 'req-9e1eda83-2bac-406b-8e58-e1ccd2b52f5b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:32.455965 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-9e1eda83-2bac-406b-8e58-e1ccd2b52f5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d70fc7f673b49868425cc9d618ba623', 'request_id': 'req-9e1eda83-2bac-406b-8e58-e1ccd2b52f5b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:32.469782 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-9e1eda83-2bac-406b-8e58-e1ccd2b52f5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d70fc7f673b49868425cc9d618ba623', 'request_id': 'req-9e1eda83-2bac-406b-8e58-e1ccd2b52f5b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:32.532721 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-9e1eda83-2bac-406b-8e58-e1ccd2b52f5b admin admin] The token cache is being invalidate because project 1d70fc7f673b49868425cc9d618ba623 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:42:32.533010 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-9e1eda83-2bac-406b-8e58-e1ccd2b52f5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:32.534964 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-9e1eda83-2bac-406b-8e58-e1ccd2b52f5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:32.536771 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 927/1875] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:42:32 2022] DELETE /identity/v3/projects/1d70fc7f673b49868425cc9d618ba623 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:42:32.541470 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08144d-1ac3-4e98-a2c5-fb02cdfa04c0 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:32.541942 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08144d-1ac3-4e98-a2c5-fb02cdfa04c0 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:32.542028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08144d-1ac3-4e98-a2c5-fb02cdfa04c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:32.562217 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8c08144d-1ac3-4e98-a2c5-fb02cdfa04c0 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:32.563411 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8c08144d-1ac3-4e98-a2c5-fb02cdfa04c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:32.627116 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 949/1876] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:32.634999 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f29e2-a355-41b0-8841-af006903795c None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:32.635318 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f29e2-a355-41b0-8841-af006903795c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:32.635647 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f29e2-a355-41b0-8841-af006903795c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:32.656052 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-328f29e2-a355-41b0-8841-af006903795c None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:32.657442 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-328f29e2-a355-41b0-8841-af006903795c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:32.665417 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 928/1877] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:32.669681 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08144d-1ac3-4e98-a2c5-fb02cdfa04c0 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:32.685700 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08144d-1ac3-4e98-a2c5-fb02cdfa04c0 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:32.694195 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d854ad0-aad1-4ca3-9a08-cb23040f44d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:32.694745 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d854ad0-aad1-4ca3-9a08-cb23040f44d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:32.695025 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d854ad0-aad1-4ca3-9a08-cb23040f44d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:32.695298 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d854ad0-aad1-4ca3-9a08-cb23040f44d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:32.695954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d854ad0-aad1-4ca3-9a08-cb23040f44d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:42:32.696254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d854ad0-aad1-4ca3-9a08-cb23040f44d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:32.697914 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d854ad0-aad1-4ca3-9a08-cb23040f44d8 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:32.713186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 950/1878] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:42:32.715817 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f29e2-a355-41b0-8841-af006903795c None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:32.726486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f29e2-a355-41b0-8841-af006903795c None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:32.731163 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb2866-8daf-40a1-a84a-2c0223fbeba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:32.731796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb2866-8daf-40a1-a84a-2c0223fbeba3 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:32.732133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb2866-8daf-40a1-a84a-2c0223fbeba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:32.732387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb2866-8daf-40a1-a84a-2c0223fbeba3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:32.733054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb2866-8daf-40a1-a84a-2c0223fbeba3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:32.734974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb2866-8daf-40a1-a84a-2c0223fbeba3 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:32.797536 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 929/1879] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:42:32 2022] POST /identity/v3/projects => generated 381 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:42:32.802983 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d854ad0-aad1-4ca3-9a08-cb23040f44d8 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:32.813498 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d854ad0-aad1-4ca3-9a08-cb23040f44d8 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:32.881437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f477f9-5faa-444a-a146-43db1e22fb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:32.882369 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f477f9-5faa-444a-a146-43db1e22fb9a admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:32.882823 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f477f9-5faa-444a-a146-43db1e22fb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:32.883275 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f477f9-5faa-444a-a146-43db1e22fb9a admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:32.884100 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f477f9-5faa-444a-a146-43db1e22fb9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:32.886248 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f477f9-5faa-444a-a146-43db1e22fb9a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:32.978425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 951/1880] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:42:32 2022] POST /identity/v3/users => generated 291 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:42:32.979972 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb2866-8daf-40a1-a84a-2c0223fbeba3 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:32.994877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb2866-8daf-40a1-a84a-2c0223fbeba3 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:32.999544 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28bc2f5-a58f-4e1d-a836-1348551f35d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:33.000178 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28bc2f5-a58f-4e1d-a836-1348551f35d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:33.000458 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28bc2f5-a58f-4e1d-a836-1348551f35d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:33.000731 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28bc2f5-a58f-4e1d-a836-1348551f35d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:33.001538 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28bc2f5-a58f-4e1d-a836-1348551f35d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:33.003141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28bc2f5-a58f-4e1d-a836-1348551f35d1 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:33.013796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 930/1881] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:42:33.019356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f477f9-5faa-444a-a146-43db1e22fb9a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:33.029323 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f477f9-5faa-444a-a146-43db1e22fb9a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:33.034507 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a71c02f-f4d1-4b75-b187-9dda85c52b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:33.035100 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71c02f-f4d1-4b75-b187-9dda85c52b98 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:33.035381 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71c02f-f4d1-4b75-b187-9dda85c52b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:33.035650 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71c02f-f4d1-4b75-b187-9dda85c52b98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:33.036288 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a71c02f-f4d1-4b75-b187-9dda85c52b98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:33.037920 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a71c02f-f4d1-4b75-b187-9dda85c52b98 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:33.047981 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 952/1882] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:42:33.052233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28bc2f5-a58f-4e1d-a836-1348551f35d1 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:33.063363 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28bc2f5-a58f-4e1d-a836-1348551f35d1 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:33.068073 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085deeb-0c2d-4644-86e2-4edf9e94b132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:33.068823 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085deeb-0c2d-4644-86e2-4edf9e94b132 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:33.069260 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085deeb-0c2d-4644-86e2-4edf9e94b132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:33.069716 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085deeb-0c2d-4644-86e2-4edf9e94b132 admin admin] PATH_INFO: `/v3/projects/a9d140f2c11a4915b48bd2e600b312ff/users/59bc2c1a7e254950b70137949dd20102/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:33.085584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085deeb-0c2d-4644-86e2-4edf9e94b132 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9d140f2c11a4915b48bd2e600b312ff, user_id=59bc2c1a7e254950b70137949dd20102, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:33.088326 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085deeb-0c2d-4644-86e2-4edf9e94b132 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:33.181137 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 931/1883] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:42:33 2022] PUT /identity/v3/projects/a9d140f2c11a4915b48bd2e600b312ff/users/59bc2c1a7e254950b70137949dd20102/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:42:33.191678 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdbd3d1-724f-4b77-b76c-eaecf7f74b29 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:33.192029 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdbd3d1-724f-4b77-b76c-eaecf7f74b29 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:33.192236 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdbd3d1-724f-4b77-b76c-eaecf7f74b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:33.226384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ecdbd3d1-724f-4b77-b76c-eaecf7f74b29 None None] MFA Rules not processed for user `59bc2c1a7e254950b70137949dd20102`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:33.227601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ecdbd3d1-724f-4b77-b76c-eaecf7f74b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:33.260489 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:33.294734 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 953/1884] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:33.299275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085deeb-0c2d-4644-86e2-4edf9e94b132 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:33.347711 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:33.362746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085deeb-0c2d-4644-86e2-4edf9e94b132 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:33.367771 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb236a50-4994-4985-a799-0c01b3f025c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:33.368669 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb236a50-4994-4985-a799-0c01b3f025c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:33.374269 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb236a50-4994-4985-a799-0c01b3f025c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:33.374269 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb236a50-4994-4985-a799-0c01b3f025c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:33.374269 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb236a50-4994-4985-a799-0c01b3f025c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:33.374269 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb236a50-4994-4985-a799-0c01b3f025c7 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:33.461026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 932/1885] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:42:33 2022] POST /identity/v3/projects => generated 381 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:42:33.465685 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdbd3d1-724f-4b77-b76c-eaecf7f74b29 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:33.481578 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdbd3d1-724f-4b77-b76c-eaecf7f74b29 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:33.555534 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0065c-5e2d-43a6-b063-ebda8bdac48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:33.556254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0065c-5e2d-43a6-b063-ebda8bdac48b admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:33.556596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0065c-5e2d-43a6-b063-ebda8bdac48b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:33.556944 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0065c-5e2d-43a6-b063-ebda8bdac48b admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:33.557693 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0065c-5e2d-43a6-b063-ebda8bdac48b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:33.559673 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0065c-5e2d-43a6-b063-ebda8bdac48b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:33.662391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 954/1886] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:42:33 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:42:33.663735 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb236a50-4994-4985-a799-0c01b3f025c7 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:33.676072 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb236a50-4994-4985-a799-0c01b3f025c7 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:33.681020 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08e0a9-abd1-47d9-aa48-519d5395d2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:33.681892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08e0a9-abd1-47d9-aa48-519d5395d2ff admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:33.682244 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08e0a9-abd1-47d9-aa48-519d5395d2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:33.682604 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08e0a9-abd1-47d9-aa48-519d5395d2ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:33.683510 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08e0a9-abd1-47d9-aa48-519d5395d2ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:33.686240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08e0a9-abd1-47d9-aa48-519d5395d2ff admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:33.704165 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 933/1887] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:42:33.705066 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0065c-5e2d-43a6-b063-ebda8bdac48b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:33.715769 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0065c-5e2d-43a6-b063-ebda8bdac48b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:33.721135 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f446ea2-3a95-48ed-b807-e3ec9d538ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:33.721822 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f446ea2-3a95-48ed-b807-e3ec9d538ba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:33.722105 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f446ea2-3a95-48ed-b807-e3ec9d538ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:33.722415 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f446ea2-3a95-48ed-b807-e3ec9d538ba8 admin admin] PATH_INFO: `/v3/projects/329e18f9ddd04f459f662b8c1c2cd6e0/users/54a8300b01114a63a228c5ccfd4c03cf/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:33.738750 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f446ea2-3a95-48ed-b807-e3ec9d538ba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=329e18f9ddd04f459f662b8c1c2cd6e0, user_id=54a8300b01114a63a228c5ccfd4c03cf, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:33.741420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f446ea2-3a95-48ed-b807-e3ec9d538ba8 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:33.811976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 955/1888] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:42:33 2022] PUT /identity/v3/projects/329e18f9ddd04f459f662b8c1c2cd6e0/users/54a8300b01114a63a228c5ccfd4c03cf/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:33.813615 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08e0a9-abd1-47d9-aa48-519d5395d2ff admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:33.825249 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08e0a9-abd1-47d9-aa48-519d5395d2ff admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:33.891714 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfdab4-d038-4464-8440-23471ff98254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:33.892327 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfdab4-d038-4464-8440-23471ff98254 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:33.892602 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfdab4-d038-4464-8440-23471ff98254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:33.892875 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfdab4-d038-4464-8440-23471ff98254 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:33.893578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfdab4-d038-4464-8440-23471ff98254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:33.895160 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfdab4-d038-4464-8440-23471ff98254 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:33.909160 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 934/1889] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:42:33.910700 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f446ea2-3a95-48ed-b807-e3ec9d538ba8 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:33.921426 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f446ea2-3a95-48ed-b807-e3ec9d538ba8 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:33.930158 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b19360f-f6b7-48c0-8a07-781d25e3f1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:33.930158 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19360f-f6b7-48c0-8a07-781d25e3f1bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:33.930158 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19360f-f6b7-48c0-8a07-781d25e3f1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:33.930158 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19360f-f6b7-48c0-8a07-781d25e3f1bd admin admin] PATH_INFO: `/v3/projects/329e18f9ddd04f459f662b8c1c2cd6e0/users/54a8300b01114a63a228c5ccfd4c03cf/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:33.932592 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b19360f-f6b7-48c0-8a07-781d25e3f1bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=329e18f9ddd04f459f662b8c1c2cd6e0, user_id=54a8300b01114a63a228c5ccfd4c03cf, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:33.935692 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b19360f-f6b7-48c0-8a07-781d25e3f1bd admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:33.947841 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 956/1890] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:42:33 2022] PUT /identity/v3/projects/329e18f9ddd04f459f662b8c1c2cd6e0/users/54a8300b01114a63a228c5ccfd4c03cf/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:33.959966 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632f150-4134-42ff-9c9e-4bb5de8fc61c None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:33.960252 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632f150-4134-42ff-9c9e-4bb5de8fc61c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:33.960537 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632f150-4134-42ff-9c9e-4bb5de8fc61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:34.006399 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-4632f150-4134-42ff-9c9e-4bb5de8fc61c None None] MFA Rules not processed for user `54a8300b01114a63a228c5ccfd4c03cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:34.007752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-4632f150-4134-42ff-9c9e-4bb5de8fc61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:34.100127 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 935/1891] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:33 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:34.115087 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b19360f-f6b7-48c0-8a07-781d25e3f1bd admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:34.117174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2b19360f-f6b7-48c0-8a07-781d25e3f1bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:34.192253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b19360f-f6b7-48c0-8a07-781d25e3f1bd admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:34.196077 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2f652-06df-4ca0-9e98-40373d78345f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:34.196686 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2f652-06df-4ca0-9e98-40373d78345f service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:34.197096 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2f652-06df-4ca0-9e98-40373d78345f service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:34.197444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2f652-06df-4ca0-9e98-40373d78345f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:34.208732 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d2f652-06df-4ca0-9e98-40373d78345f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:34.210232 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d2f652-06df-4ca0-9e98-40373d78345f service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:34.221952 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 957/1892] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:42:34 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:42:34.885350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632f150-4134-42ff-9c9e-4bb5de8fc61c None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:34.945210 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:34.964250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632f150-4134-42ff-9c9e-4bb5de8fc61c None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:34.970925 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d2758-c875-4853-ab7f-e06bb7357b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:34.971637 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d2758-c875-4853-ab7f-e06bb7357b5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:34.971856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d2758-c875-4853-ab7f-e06bb7357b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:34.972130 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d2758-c875-4853-ab7f-e06bb7357b5b admin admin] PATH_INFO: `/v3/users/59bc2c1a7e254950b70137949dd20102` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:34.979095 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d2758-c875-4853-ab7f-e06bb7357b5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=59bc2c1a7e254950b70137949dd20102)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:34.981057 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d2758-c875-4853-ab7f-e06bb7357b5b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:35.164712 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-472d2758-c875-4853-ab7f-e06bb7357b5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59bc2c1a7e254950b70137949dd20102'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:35.165504 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-472d2758-c875-4853-ab7f-e06bb7357b5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59bc2c1a7e254950b70137949dd20102'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:35.166205 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-472d2758-c875-4853-ab7f-e06bb7357b5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59bc2c1a7e254950b70137949dd20102'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:35.173168 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-472d2758-c875-4853-ab7f-e06bb7357b5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59bc2c1a7e254950b70137949dd20102'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:35.241253 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-472d2758-c875-4853-ab7f-e06bb7357b5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59bc2c1a7e254950b70137949dd20102'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:35.264413 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 936/1893] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:42:34 2022] DELETE /identity/v3/users/59bc2c1a7e254950b70137949dd20102 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:35.269167 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7a912-53bc-47c2-b05e-a3aa0c0cfc85 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:35.269658 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7a912-53bc-47c2-b05e-a3aa0c0cfc85 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:35.269750 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7a912-53bc-47c2-b05e-a3aa0c0cfc85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:35.288879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-10d7a912-53bc-47c2-b05e-a3aa0c0cfc85 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:35.290031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-10d7a912-53bc-47c2-b05e-a3aa0c0cfc85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:35.316819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:35.355772 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 958/1894] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:35.365664 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d2758-c875-4853-ab7f-e06bb7357b5b admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:35.367099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-472d2758-c875-4853-ab7f-e06bb7357b5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:35.424386 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d2758-c875-4853-ab7f-e06bb7357b5b admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:35.428032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb3f5e-bb32-4bce-93a1-80d265168dd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:35.428694 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb3f5e-bb32-4bce-93a1-80d265168dd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:35.429398 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb3f5e-bb32-4bce-93a1-80d265168dd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:35.429398 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb3f5e-bb32-4bce-93a1-80d265168dd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:35.440744 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb3f5e-bb32-4bce-93a1-80d265168dd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:35.442243 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb3f5e-bb32-4bce-93a1-80d265168dd7 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:35.452858 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 937/1895] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:42:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:42:35.854748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7a912-53bc-47c2-b05e-a3aa0c0cfc85 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:35.856617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-10d7a912-53bc-47c2-b05e-a3aa0c0cfc85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:35.870555 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7a912-53bc-47c2-b05e-a3aa0c0cfc85 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:35.877004 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd6380-4b98-40e1-90bb-19d770c3f1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:35.877793 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd6380-4b98-40e1-90bb-19d770c3f1a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:35.878145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd6380-4b98-40e1-90bb-19d770c3f1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:35.878492 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd6380-4b98-40e1-90bb-19d770c3f1a2 admin admin] PATH_INFO: `/v3/projects/a9d140f2c11a4915b48bd2e600b312ff` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:35.879780 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdd6380-4b98-40e1-90bb-19d770c3f1a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9d140f2c11a4915b48bd2e600b312ff)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:35.881752 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdd6380-4b98-40e1-90bb-19d770c3f1a2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:35.976507 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1bdd6380-4b98-40e1-90bb-19d770c3f1a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9d140f2c11a4915b48bd2e600b312ff', 'request_id': 'req-1bdd6380-4b98-40e1-90bb-19d770c3f1a2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:36.053068 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1bdd6380-4b98-40e1-90bb-19d770c3f1a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9d140f2c11a4915b48bd2e600b312ff', 'request_id': 'req-1bdd6380-4b98-40e1-90bb-19d770c3f1a2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:36.063511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1bdd6380-4b98-40e1-90bb-19d770c3f1a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9d140f2c11a4915b48bd2e600b312ff', 'request_id': 'req-1bdd6380-4b98-40e1-90bb-19d770c3f1a2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:36.073190 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1bdd6380-4b98-40e1-90bb-19d770c3f1a2 admin admin] The token cache is being invalidate because project a9d140f2c11a4915b48bd2e600b312ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:42:36.073490 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1bdd6380-4b98-40e1-90bb-19d770c3f1a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:36.074137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1bdd6380-4b98-40e1-90bb-19d770c3f1a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:36.078889 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 959/1896] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:42:35 2022] DELETE /identity/v3/projects/a9d140f2c11a4915b48bd2e600b312ff => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:36.080544 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb3f5e-bb32-4bce-93a1-80d265168dd7 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:36.091580 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb3f5e-bb32-4bce-93a1-80d265168dd7 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:36.174380 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957fb5d-d32d-46b7-8610-341bfac95575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:36.174380 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957fb5d-d32d-46b7-8610-341bfac95575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:36.174380 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957fb5d-d32d-46b7-8610-341bfac95575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:36.174380 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957fb5d-d32d-46b7-8610-341bfac95575 admin admin] PATH_INFO: `/v3/users/54a8300b01114a63a228c5ccfd4c03cf` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:36.175233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957fb5d-d32d-46b7-8610-341bfac95575 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54a8300b01114a63a228c5ccfd4c03cf)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:36.175233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957fb5d-d32d-46b7-8610-341bfac95575 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:36.179659 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:36.307118 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-3957fb5d-d32d-46b7-8610-341bfac95575 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54a8300b01114a63a228c5ccfd4c03cf'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:36.307762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-3957fb5d-d32d-46b7-8610-341bfac95575 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54a8300b01114a63a228c5ccfd4c03cf'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:36.308362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-3957fb5d-d32d-46b7-8610-341bfac95575 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54a8300b01114a63a228c5ccfd4c03cf'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:36.315224 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-3957fb5d-d32d-46b7-8610-341bfac95575 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54a8300b01114a63a228c5ccfd4c03cf'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:36.390861 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-3957fb5d-d32d-46b7-8610-341bfac95575 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54a8300b01114a63a228c5ccfd4c03cf'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:36.409894 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 938/1897] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:42:36 2022] DELETE /identity/v3/users/54a8300b01114a63a228c5ccfd4c03cf => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:36.836609 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd6380-4b98-40e1-90bb-19d770c3f1a2 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:36.838344 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1bdd6380-4b98-40e1-90bb-19d770c3f1a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:36.857664 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd6380-4b98-40e1-90bb-19d770c3f1a2 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:36.917065 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9ca3b-948a-458e-b77b-8bd7f26d2648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:36.917662 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9ca3b-948a-458e-b77b-8bd7f26d2648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:36.917942 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9ca3b-948a-458e-b77b-8bd7f26d2648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:36.918217 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9ca3b-948a-458e-b77b-8bd7f26d2648 admin admin] PATH_INFO: `/v3/projects/329e18f9ddd04f459f662b8c1c2cd6e0` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:36.919331 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d9ca3b-948a-458e-b77b-8bd7f26d2648 admin admin] RBAC: Authorizing `identity:delete_project(project_id=329e18f9ddd04f459f662b8c1c2cd6e0)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:36.921292 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d9ca3b-948a-458e-b77b-8bd7f26d2648 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:37.042436 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c7d9ca3b-948a-458e-b77b-8bd7f26d2648 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '329e18f9ddd04f459f662b8c1c2cd6e0', 'request_id': 'req-c7d9ca3b-948a-458e-b77b-8bd7f26d2648'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:37.046466 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:37.155828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c7d9ca3b-948a-458e-b77b-8bd7f26d2648 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '329e18f9ddd04f459f662b8c1c2cd6e0', 'request_id': 'req-c7d9ca3b-948a-458e-b77b-8bd7f26d2648'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:37.163519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c7d9ca3b-948a-458e-b77b-8bd7f26d2648 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '329e18f9ddd04f459f662b8c1c2cd6e0', 'request_id': 'req-c7d9ca3b-948a-458e-b77b-8bd7f26d2648'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:37.189753 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c7d9ca3b-948a-458e-b77b-8bd7f26d2648 admin admin] The token cache is being invalidate because project 329e18f9ddd04f459f662b8c1c2cd6e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:42:37.190051 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c7d9ca3b-948a-458e-b77b-8bd7f26d2648 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:37.190763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c7d9ca3b-948a-458e-b77b-8bd7f26d2648 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:37.193099 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 960/1898] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:42:36 2022] DELETE /identity/v3/projects/329e18f9ddd04f459f662b8c1c2cd6e0 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:37.197986 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ad68d-075c-407b-8115-13cf9d26097e None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:37.198503 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ad68d-075c-407b-8115-13cf9d26097e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:37.198618 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ad68d-075c-407b-8115-13cf9d26097e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:37.219660 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-6f0ad68d-075c-407b-8115-13cf9d26097e None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:37.220872 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6f0ad68d-075c-407b-8115-13cf9d26097e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:37.282514 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 939/1899] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:37.290442 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91640b-3e6d-4bdc-a138-8bb509b9ddb5 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:37.290759 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91640b-3e6d-4bdc-a138-8bb509b9ddb5 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:37.291175 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91640b-3e6d-4bdc-a138-8bb509b9ddb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:37.312816 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-9e91640b-3e6d-4bdc-a138-8bb509b9ddb5 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:37.314054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9e91640b-3e6d-4bdc-a138-8bb509b9ddb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:37.321788 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 961/1900] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:37.325873 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ad68d-075c-407b-8115-13cf9d26097e None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:37.335702 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ad68d-075c-407b-8115-13cf9d26097e None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:37.340222 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdda8a2-fc39-4e21-9f61-3ed57ada8eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:37.340819 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdda8a2-fc39-4e21-9f61-3ed57ada8eed admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:37.341098 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdda8a2-fc39-4e21-9f61-3ed57ada8eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:37.341401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdda8a2-fc39-4e21-9f61-3ed57ada8eed admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:37.342044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdda8a2-fc39-4e21-9f61-3ed57ada8eed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:42:37.342345 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdda8a2-fc39-4e21-9f61-3ed57ada8eed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:37.344094 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdda8a2-fc39-4e21-9f61-3ed57ada8eed admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:37.355563 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 940/1901] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:42:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:42:37.360337 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e91640b-3e6d-4bdc-a138-8bb509b9ddb5 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:37.369685 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e91640b-3e6d-4bdc-a138-8bb509b9ddb5 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:37.375122 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53722b45-fe80-4f86-b133-31a752ab8c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:37.375709 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53722b45-fe80-4f86-b133-31a752ab8c60 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:37.375985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53722b45-fe80-4f86-b133-31a752ab8c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:37.376251 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53722b45-fe80-4f86-b133-31a752ab8c60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:37.376857 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53722b45-fe80-4f86-b133-31a752ab8c60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:37.378774 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53722b45-fe80-4f86-b133-31a752ab8c60 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:37.464861 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 962/1902] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:42:37 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:42:37.469164 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdda8a2-fc39-4e21-9f61-3ed57ada8eed admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:37.479489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdda8a2-fc39-4e21-9f61-3ed57ada8eed admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:37.534752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27273c54-d30f-4915-b96b-878faf9a46f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:37.535347 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27273c54-d30f-4915-b96b-878faf9a46f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:37.535624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27273c54-d30f-4915-b96b-878faf9a46f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:37.535894 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27273c54-d30f-4915-b96b-878faf9a46f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:37.536511 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27273c54-d30f-4915-b96b-878faf9a46f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:37.538425 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27273c54-d30f-4915-b96b-878faf9a46f0 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:37.607708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:37.613687 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 941/1903] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:42:37 2022] POST /identity/v3/users => generated 286 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:42:37.617732 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53722b45-fe80-4f86-b133-31a752ab8c60 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:37.628193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53722b45-fe80-4f86-b133-31a752ab8c60 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:37.632654 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30609d6-5bc4-41ca-9e0d-4d40fc245708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:37.633282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30609d6-5bc4-41ca-9e0d-4d40fc245708 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:37.633604 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30609d6-5bc4-41ca-9e0d-4d40fc245708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:37.633872 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30609d6-5bc4-41ca-9e0d-4d40fc245708 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:37.634509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30609d6-5bc4-41ca-9e0d-4d40fc245708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:37.636086 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30609d6-5bc4-41ca-9e0d-4d40fc245708 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:37.646229 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 963/1904] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:42:37.650240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27273c54-d30f-4915-b96b-878faf9a46f0 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:37.661168 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27273c54-d30f-4915-b96b-878faf9a46f0 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:37.666669 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d374371-1f98-44e2-8a7f-0d9aefe3f4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:37.667262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d374371-1f98-44e2-8a7f-0d9aefe3f4b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:37.667539 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d374371-1f98-44e2-8a7f-0d9aefe3f4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:37.668062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d374371-1f98-44e2-8a7f-0d9aefe3f4b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:37.668701 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d374371-1f98-44e2-8a7f-0d9aefe3f4b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:37.683073 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d374371-1f98-44e2-8a7f-0d9aefe3f4b7 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:37.683073 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 942/1905] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:42:37.686667 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30609d6-5bc4-41ca-9e0d-4d40fc245708 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:37.696748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30609d6-5bc4-41ca-9e0d-4d40fc245708 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:37.701438 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e04ca-1895-420d-859c-1c108a588474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:37.702212 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e04ca-1895-420d-859c-1c108a588474 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:37.702425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e04ca-1895-420d-859c-1c108a588474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:37.702699 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e04ca-1895-420d-859c-1c108a588474 admin admin] PATH_INFO: `/v3/projects/3e90ac16a3144b9baa7a77306e1f56c0/users/4e45e73e1bf548f5909509403c123785/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:37.718590 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e04ca-1895-420d-859c-1c108a588474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e90ac16a3144b9baa7a77306e1f56c0, user_id=4e45e73e1bf548f5909509403c123785, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:37.721270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e04ca-1895-420d-859c-1c108a588474 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:37.789610 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 964/1906] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:42:37 2022] PUT /identity/v3/projects/3e90ac16a3144b9baa7a77306e1f56c0/users/4e45e73e1bf548f5909509403c123785/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:37.801741 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca61ea-546f-40d3-90c4-1cc760c4c6c6 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:37.802029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca61ea-546f-40d3-90c4-1cc760c4c6c6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:37.802305 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca61ea-546f-40d3-90c4-1cc760c4c6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:37.832259 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-57ca61ea-546f-40d3-90c4-1cc760c4c6c6 None None] MFA Rules not processed for user `4e45e73e1bf548f5909509403c123785`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:37.833514 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-57ca61ea-546f-40d3-90c4-1cc760c4c6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:37.904979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 943/1907] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:37.916273 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e04ca-1895-420d-859c-1c108a588474 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:37.918022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e63e04ca-1895-420d-859c-1c108a588474 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:37.976561 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e04ca-1895-420d-859c-1c108a588474 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:37.980101 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de820a3a-e6fd-43bd-9051-22e5bd70d2ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:37.980715 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de820a3a-e6fd-43bd-9051-22e5bd70d2ed service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:37.981021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de820a3a-e6fd-43bd-9051-22e5bd70d2ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:37.981345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de820a3a-e6fd-43bd-9051-22e5bd70d2ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:37.990371 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de820a3a-e6fd-43bd-9051-22e5bd70d2ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:37.991817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de820a3a-e6fd-43bd-9051-22e5bd70d2ed service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:38.001844 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 965/1908] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:42:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:42:38.022966 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca61ea-546f-40d3-90c4-1cc760c4c6c6 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:38.086889 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca61ea-546f-40d3-90c4-1cc760c4c6c6 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:38.091467 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbda86d-76d7-4490-835f-ba29989120b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:38.092092 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbda86d-76d7-4490-835f-ba29989120b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:38.092373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbda86d-76d7-4490-835f-ba29989120b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:38.093305 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbda86d-76d7-4490-835f-ba29989120b1 admin admin] PATH_INFO: `/v3/users/4e45e73e1bf548f5909509403c123785` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:38.095512 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbda86d-76d7-4490-835f-ba29989120b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e45e73e1bf548f5909509403c123785)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:38.096363 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbda86d-76d7-4490-835f-ba29989120b1 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:38.297777 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ccbda86d-76d7-4490-835f-ba29989120b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e45e73e1bf548f5909509403c123785'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:38.301580 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ccbda86d-76d7-4490-835f-ba29989120b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e45e73e1bf548f5909509403c123785'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:38.302232 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ccbda86d-76d7-4490-835f-ba29989120b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e45e73e1bf548f5909509403c123785'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:38.309408 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ccbda86d-76d7-4490-835f-ba29989120b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e45e73e1bf548f5909509403c123785'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:38.349976 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ccbda86d-76d7-4490-835f-ba29989120b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e45e73e1bf548f5909509403c123785'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:38.364631 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:38.369635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 944/1909] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:42:38 2022] DELETE /identity/v3/users/4e45e73e1bf548f5909509403c123785 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:38.374076 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857c342-f19e-43a7-ae23-dcc863103f67 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:38.374363 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857c342-f19e-43a7-ae23-dcc863103f67 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:38.374634 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857c342-f19e-43a7-ae23-dcc863103f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:38.393845 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-6857c342-f19e-43a7-ae23-dcc863103f67 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:38.395084 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6857c342-f19e-43a7-ae23-dcc863103f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:38.458185 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 966/1910] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:38.468338 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbda86d-76d7-4490-835f-ba29989120b1 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:38.469730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ccbda86d-76d7-4490-835f-ba29989120b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:38.524970 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbda86d-76d7-4490-835f-ba29989120b1 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:38.528452 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e28a5-f55c-460d-8214-40934d42c845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:38.529048 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e28a5-f55c-460d-8214-40934d42c845 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:38.529347 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e28a5-f55c-460d-8214-40934d42c845 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:38.529616 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e28a5-f55c-460d-8214-40934d42c845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:38.538923 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e28a5-f55c-460d-8214-40934d42c845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:38.540321 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e28a5-f55c-460d-8214-40934d42c845 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:38.550502 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 945/1911] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:42:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:42:38.912053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857c342-f19e-43a7-ae23-dcc863103f67 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:38.913653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6857c342-f19e-43a7-ae23-dcc863103f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:38.926334 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857c342-f19e-43a7-ae23-dcc863103f67 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:38.931245 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6effc3-a37a-4506-9032-8f5ff7a042ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:38.931955 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6effc3-a37a-4506-9032-8f5ff7a042ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:38.932299 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6effc3-a37a-4506-9032-8f5ff7a042ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:38.932658 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6effc3-a37a-4506-9032-8f5ff7a042ee admin admin] PATH_INFO: `/v3/projects/3e90ac16a3144b9baa7a77306e1f56c0` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:38.933946 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6effc3-a37a-4506-9032-8f5ff7a042ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e90ac16a3144b9baa7a77306e1f56c0)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:38.935902 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6effc3-a37a-4506-9032-8f5ff7a042ee admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:38.953495 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:39.088985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5a6effc3-a37a-4506-9032-8f5ff7a042ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e90ac16a3144b9baa7a77306e1f56c0', 'request_id': 'req-5a6effc3-a37a-4506-9032-8f5ff7a042ee'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:39.167034 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5a6effc3-a37a-4506-9032-8f5ff7a042ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e90ac16a3144b9baa7a77306e1f56c0', 'request_id': 'req-5a6effc3-a37a-4506-9032-8f5ff7a042ee'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:39.173917 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5a6effc3-a37a-4506-9032-8f5ff7a042ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e90ac16a3144b9baa7a77306e1f56c0', 'request_id': 'req-5a6effc3-a37a-4506-9032-8f5ff7a042ee'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:39.185040 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5a6effc3-a37a-4506-9032-8f5ff7a042ee admin admin] The token cache is being invalidate because project 3e90ac16a3144b9baa7a77306e1f56c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:42:39.185382 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5a6effc3-a37a-4506-9032-8f5ff7a042ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:39.191042 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5a6effc3-a37a-4506-9032-8f5ff7a042ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:39.191042 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 967/1912] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:42:38 2022] DELETE /identity/v3/projects/3e90ac16a3144b9baa7a77306e1f56c0 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:39.192975 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501453d5-79c3-46b3-93b0-798ccdbeb6dd None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:39.193323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501453d5-79c3-46b3-93b0-798ccdbeb6dd None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:39.193672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501453d5-79c3-46b3-93b0-798ccdbeb6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:39.213764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-501453d5-79c3-46b3-93b0-798ccdbeb6dd None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:39.214959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-501453d5-79c3-46b3-93b0-798ccdbeb6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:39.273291 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 946/1913] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:39.281653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e75be4-35ed-42e3-be8f-780586d7e482 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:39.281942 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e75be4-35ed-42e3-be8f-780586d7e482 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:39.282217 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e75be4-35ed-42e3-be8f-780586d7e482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:39.303230 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-78e75be4-35ed-42e3-be8f-780586d7e482 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:39.304468 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-78e75be4-35ed-42e3-be8f-780586d7e482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:39.316238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 968/1914] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:39.318653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501453d5-79c3-46b3-93b0-798ccdbeb6dd None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:39.329193 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501453d5-79c3-46b3-93b0-798ccdbeb6dd None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:39.333834 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d9c31-022e-429c-976b-f9123a96e47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:39.334441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d9c31-022e-429c-976b-f9123a96e47a admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:39.334724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d9c31-022e-429c-976b-f9123a96e47a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:39.335001 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d9c31-022e-429c-976b-f9123a96e47a admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:39.335654 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d9c31-022e-429c-976b-f9123a96e47a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:42:39.335977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d9c31-022e-429c-976b-f9123a96e47a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:39.337594 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d9c31-022e-429c-976b-f9123a96e47a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:39.351027 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 947/1915] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:42:39.355280 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e75be4-35ed-42e3-be8f-780586d7e482 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:39.364615 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e75be4-35ed-42e3-be8f-780586d7e482 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:39.369132 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ab88f-c394-4231-a4b5-cf22ef2c7ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:39.369833 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ab88f-c394-4231-a4b5-cf22ef2c7ccc admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:39.370114 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ab88f-c394-4231-a4b5-cf22ef2c7ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:39.370385 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ab88f-c394-4231-a4b5-cf22ef2c7ccc admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:39.371014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ab88f-c394-4231-a4b5-cf22ef2c7ccc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:39.372892 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ab88f-c394-4231-a4b5-cf22ef2c7ccc admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:39.466022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 969/1916] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:42:39 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:42:39.470183 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d9c31-022e-429c-976b-f9123a96e47a admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:39.480768 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d9c31-022e-429c-976b-f9123a96e47a admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:39.543695 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4855399-0449-472e-8fe8-bedfd88195aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:39.544312 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4855399-0449-472e-8fe8-bedfd88195aa admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:39.544592 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4855399-0449-472e-8fe8-bedfd88195aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:39.544864 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4855399-0449-472e-8fe8-bedfd88195aa admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:39.545542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4855399-0449-472e-8fe8-bedfd88195aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:39.548008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4855399-0449-472e-8fe8-bedfd88195aa admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:39.633209 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 948/1917] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:42:39 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:42:39.635208 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ab88f-c394-4231-a4b5-cf22ef2c7ccc admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:39.645605 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ab88f-c394-4231-a4b5-cf22ef2c7ccc admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:39.650102 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d262e6-e977-447b-978d-4fcd5a115eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:39.650694 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d262e6-e977-447b-978d-4fcd5a115eef admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:39.650972 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d262e6-e977-447b-978d-4fcd5a115eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:39.651266 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d262e6-e977-447b-978d-4fcd5a115eef admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:39.653525 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d262e6-e977-447b-978d-4fcd5a115eef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:39.653525 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d262e6-e977-447b-978d-4fcd5a115eef admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:39.663347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 970/1918] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:42:39.667398 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4855399-0449-472e-8fe8-bedfd88195aa admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:39.678278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4855399-0449-472e-8fe8-bedfd88195aa admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:39.682919 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195dfa56-67a0-4e27-bb26-2023a0263f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:39.683509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195dfa56-67a0-4e27-bb26-2023a0263f0e admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:39.683789 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195dfa56-67a0-4e27-bb26-2023a0263f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:39.684061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195dfa56-67a0-4e27-bb26-2023a0263f0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:39.684698 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195dfa56-67a0-4e27-bb26-2023a0263f0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:39.686342 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195dfa56-67a0-4e27-bb26-2023a0263f0e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:39.697495 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 949/1919] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:42:39.701633 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d262e6-e977-447b-978d-4fcd5a115eef admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:39.711731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d262e6-e977-447b-978d-4fcd5a115eef admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:39.716616 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e76b18-8ae8-41fc-a883-5fef12b090fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:39.717349 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e76b18-8ae8-41fc-a883-5fef12b090fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:39.717627 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e76b18-8ae8-41fc-a883-5fef12b090fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:39.717891 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e76b18-8ae8-41fc-a883-5fef12b090fc admin admin] PATH_INFO: `/v3/projects/7b33311164464811836d22bdf73b800a/users/8abe7d184c124110bb39780bc61b294d/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:39.732086 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e76b18-8ae8-41fc-a883-5fef12b090fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b33311164464811836d22bdf73b800a, user_id=8abe7d184c124110bb39780bc61b294d, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:39.735310 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e76b18-8ae8-41fc-a883-5fef12b090fc admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:39.807408 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 971/1920] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:42:39 2022] PUT /identity/v3/projects/7b33311164464811836d22bdf73b800a/users/8abe7d184c124110bb39780bc61b294d/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:39.818026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdc4b6-be36-4b15-bbba-ea25a5d3e6c6 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:39.818319 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdc4b6-be36-4b15-bbba-ea25a5d3e6c6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:39.818595 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdc4b6-be36-4b15-bbba-ea25a5d3e6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:39.831573 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:39.846169 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-acbdc4b6-be36-4b15-bbba-ea25a5d3e6c6 None None] MFA Rules not processed for user `8abe7d184c124110bb39780bc61b294d`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:39.847402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-acbdc4b6-be36-4b15-bbba-ea25a5d3e6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:39.911082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 950/1921] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:39.915597 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e76b18-8ae8-41fc-a883-5fef12b090fc admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:39.997989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e76b18-8ae8-41fc-a883-5fef12b090fc admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:40.002837 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323bf94-1b19-43fd-8341-4c8e68809af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:40.003445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323bf94-1b19-43fd-8341-4c8e68809af4 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:40.003727 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323bf94-1b19-43fd-8341-4c8e68809af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:40.003995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323bf94-1b19-43fd-8341-4c8e68809af4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:40.004624 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1323bf94-1b19-43fd-8341-4c8e68809af4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:40.006617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1323bf94-1b19-43fd-8341-4c8e68809af4 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:40.082388 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 972/1922] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:42:39 2022] POST /identity/v3/projects => generated 385 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:42:40.086443 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbdc4b6-be36-4b15-bbba-ea25a5d3e6c6 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:40.097724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbdc4b6-be36-4b15-bbba-ea25a5d3e6c6 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:40.154732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a004f8ce-cafe-42eb-8f24-374e878196cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:40.155961 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004f8ce-cafe-42eb-8f24-374e878196cf admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:40.156236 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004f8ce-cafe-42eb-8f24-374e878196cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:40.156508 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004f8ce-cafe-42eb-8f24-374e878196cf admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:40.157132 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a004f8ce-cafe-42eb-8f24-374e878196cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:40.159054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a004f8ce-cafe-42eb-8f24-374e878196cf admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:40.269787 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 951/1923] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:42:40 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:42:40.273828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323bf94-1b19-43fd-8341-4c8e68809af4 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:40.285049 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323bf94-1b19-43fd-8341-4c8e68809af4 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:40.289733 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372ca54-2a04-41f5-a126-a45b60713683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:40.290383 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372ca54-2a04-41f5-a126-a45b60713683 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:40.290667 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372ca54-2a04-41f5-a126-a45b60713683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:40.290941 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372ca54-2a04-41f5-a126-a45b60713683 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:40.291616 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8372ca54-2a04-41f5-a126-a45b60713683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:40.293256 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8372ca54-2a04-41f5-a126-a45b60713683 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:40.304190 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 973/1924] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:42:40.308059 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a004f8ce-cafe-42eb-8f24-374e878196cf admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:40.318238 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a004f8ce-cafe-42eb-8f24-374e878196cf admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:40.323106 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ec9ff-2ddf-430e-a5d4-0880014da755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:40.323691 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ec9ff-2ddf-430e-a5d4-0880014da755 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:40.323969 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ec9ff-2ddf-430e-a5d4-0880014da755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:40.324242 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ec9ff-2ddf-430e-a5d4-0880014da755 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:40.324882 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ec9ff-2ddf-430e-a5d4-0880014da755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:40.326536 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ec9ff-2ddf-430e-a5d4-0880014da755 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:40.337285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 952/1925] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:42:40.341318 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372ca54-2a04-41f5-a126-a45b60713683 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:40.352559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372ca54-2a04-41f5-a126-a45b60713683 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:40.357426 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17612d5-ef1e-4771-8013-e167628e816c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:40.358091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17612d5-ef1e-4771-8013-e167628e816c admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:40.358498 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17612d5-ef1e-4771-8013-e167628e816c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:40.359051 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17612d5-ef1e-4771-8013-e167628e816c admin admin] PATH_INFO: `/v3/projects/2f735573634743bca32ff83b049e299a/users/9299911c4f1747c4a44134a4aac1f01c/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:40.372861 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:40.376193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17612d5-ef1e-4771-8013-e167628e816c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f735573634743bca32ff83b049e299a, user_id=9299911c4f1747c4a44134a4aac1f01c, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:40.378964 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17612d5-ef1e-4771-8013-e167628e816c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:40.470980 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 974/1926] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:42:40 2022] PUT /identity/v3/projects/2f735573634743bca32ff83b049e299a/users/9299911c4f1747c4a44134a4aac1f01c/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:40.482251 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5e76b-d9c5-4bd0-9b40-e4951f5f7e2e None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:40.482773 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5e76b-d9c5-4bd0-9b40-e4951f5f7e2e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:40.483123 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5e76b-d9c5-4bd0-9b40-e4951f5f7e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:40.513374 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-15d5e76b-d9c5-4bd0-9b40-e4951f5f7e2e None None] MFA Rules not processed for user `9299911c4f1747c4a44134a4aac1f01c`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:40.515979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-15d5e76b-d9c5-4bd0-9b40-e4951f5f7e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:40.584728 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 953/1927] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:42:40.589484 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17612d5-ef1e-4771-8013-e167628e816c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:40.657526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17612d5-ef1e-4771-8013-e167628e816c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:40.662551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5e0c4-3a3e-42b2-b84a-45d786d36686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:40.663174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5e0c4-3a3e-42b2-b84a-45d786d36686 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:40.663447 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5e0c4-3a3e-42b2-b84a-45d786d36686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:40.663771 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5e0c4-3a3e-42b2-b84a-45d786d36686 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:40.664912 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5e0c4-3a3e-42b2-b84a-45d786d36686 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:40.666380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5e0c4-3a3e-42b2-b84a-45d786d36686 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:40.739727 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 975/1928] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:42:40 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:42:40.743977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5e76b-d9c5-4bd0-9b40-e4951f5f7e2e None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:40.750803 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:40.755091 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5e76b-d9c5-4bd0-9b40-e4951f5f7e2e None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:40.811319 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e175c-9983-4276-b497-45e86305251b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:40.811925 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e175c-9983-4276-b497-45e86305251b admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:40.812203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e175c-9983-4276-b497-45e86305251b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:40.812471 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e175c-9983-4276-b497-45e86305251b admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:40.813096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e175c-9983-4276-b497-45e86305251b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:40.815455 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e175c-9983-4276-b497-45e86305251b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:40.885864 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 954/1929] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:42:40 2022] POST /identity/v3/users => generated 299 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:42:40.890356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5e0c4-3a3e-42b2-b84a-45d786d36686 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:40.900500 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5e0c4-3a3e-42b2-b84a-45d786d36686 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:40.906068 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee125e04-7e81-4543-b482-8023a4edea1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:40.906918 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee125e04-7e81-4543-b482-8023a4edea1c admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:40.907195 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee125e04-7e81-4543-b482-8023a4edea1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:40.907462 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee125e04-7e81-4543-b482-8023a4edea1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:40.908098 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee125e04-7e81-4543-b482-8023a4edea1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:40.909743 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee125e04-7e81-4543-b482-8023a4edea1c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:40.920068 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 976/1930] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:42:40.924289 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e175c-9983-4276-b497-45e86305251b admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:40.934603 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e175c-9983-4276-b497-45e86305251b admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:40.939359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7936a-60c4-4c1c-ad81-cb08baeebd75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:40.940003 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7936a-60c4-4c1c-ad81-cb08baeebd75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:40.940281 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7936a-60c4-4c1c-ad81-cb08baeebd75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:40.940552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7936a-60c4-4c1c-ad81-cb08baeebd75 admin admin] PATH_INFO: `/v3/projects/a29562a2853c41afa064bda60e1f72be/users/c3c1606bbf73426883b588131fef6ad4/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:40.956105 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7936a-60c4-4c1c-ad81-cb08baeebd75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a29562a2853c41afa064bda60e1f72be, user_id=c3c1606bbf73426883b588131fef6ad4, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:40.958816 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7936a-60c4-4c1c-ad81-cb08baeebd75 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:41.051623 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 955/1931] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:42:40 2022] PUT /identity/v3/projects/a29562a2853c41afa064bda60e1f72be/users/c3c1606bbf73426883b588131fef6ad4/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:41.055762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee125e04-7e81-4543-b482-8023a4edea1c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:41.065904 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee125e04-7e81-4543-b482-8023a4edea1c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:41.121287 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e54fb-f122-476b-85f9-623c2a46b755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:41.121900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e54fb-f122-476b-85f9-623c2a46b755 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:41.122178 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e54fb-f122-476b-85f9-623c2a46b755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:41.122443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e54fb-f122-476b-85f9-623c2a46b755 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:41.123075 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e54fb-f122-476b-85f9-623c2a46b755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:41.124889 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e54fb-f122-476b-85f9-623c2a46b755 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:41.134473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 977/1932] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:42:41.138578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7936a-60c4-4c1c-ad81-cb08baeebd75 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:41.149061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7936a-60c4-4c1c-ad81-cb08baeebd75 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:41.153551 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2d9e8-db09-46cc-b4e8-4cb7c617115a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:41.154191 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2d9e8-db09-46cc-b4e8-4cb7c617115a admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:41.154466 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2d9e8-db09-46cc-b4e8-4cb7c617115a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:41.154734 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2d9e8-db09-46cc-b4e8-4cb7c617115a admin admin] PATH_INFO: `/v3/projects/a29562a2853c41afa064bda60e1f72be/users/c3c1606bbf73426883b588131fef6ad4/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:41.157432 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e2d9e8-db09-46cc-b4e8-4cb7c617115a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a29562a2853c41afa064bda60e1f72be, user_id=c3c1606bbf73426883b588131fef6ad4, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:41.160008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e2d9e8-db09-46cc-b4e8-4cb7c617115a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:41.171169 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 956/1933] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:42:41 2022] PUT /identity/v3/projects/a29562a2853c41afa064bda60e1f72be/users/c3c1606bbf73426883b588131fef6ad4/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:41.181277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6dd20-c9d7-4020-be57-f2387bfb2f8a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:41.181561 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6dd20-c9d7-4020-be57-f2387bfb2f8a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:41.181827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6dd20-c9d7-4020-be57-f2387bfb2f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:41.209123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8aa6dd20-c9d7-4020-be57-f2387bfb2f8a None None] MFA Rules not processed for user `c3c1606bbf73426883b588131fef6ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:41.210329 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8aa6dd20-c9d7-4020-be57-f2387bfb2f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:41.258026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:41.278385 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 978/1934] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:41.289940 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2d9e8-db09-46cc-b4e8-4cb7c617115a admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:41.291689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-70e2d9e8-db09-46cc-b4e8-4cb7c617115a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:41.347872 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2d9e8-db09-46cc-b4e8-4cb7c617115a admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:41.351697 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f3588-fbf5-4f0c-be97-76fb6d03301a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:41.352289 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f3588-fbf5-4f0c-be97-76fb6d03301a service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:41.352566 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f3588-fbf5-4f0c-be97-76fb6d03301a service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:41.352836 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f3588-fbf5-4f0c-be97-76fb6d03301a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:41.412813 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f3588-fbf5-4f0c-be97-76fb6d03301a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:41.414299 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f3588-fbf5-4f0c-be97-76fb6d03301a service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:41.424321 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 957/1935] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:42:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:42:45.653801 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6dd20-c9d7-4020-be57-f2387bfb2f8a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:45.665832 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6dd20-c9d7-4020-be57-f2387bfb2f8a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:45.669736 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e87f636-6930-4256-883e-5f6006dd9e86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:45.670349 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87f636-6930-4256-883e-5f6006dd9e86 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:45.670628 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87f636-6930-4256-883e-5f6006dd9e86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:45.670900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87f636-6930-4256-883e-5f6006dd9e86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:45.730710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e87f636-6930-4256-883e-5f6006dd9e86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:45.732161 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e87f636-6930-4256-883e-5f6006dd9e86 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:45.741981 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 979/1936] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:42:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:42:47.266206 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f3588-fbf5-4f0c-be97-76fb6d03301a service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:47.274573 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f3588-fbf5-4f0c-be97-76fb6d03301a service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:47.278159 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc8a7c-48a6-43c4-84f9-770e6f3fc98a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:47.278750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc8a7c-48a6-43c4-84f9-770e6f3fc98a service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:47.279031 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc8a7c-48a6-43c4-84f9-770e6f3fc98a service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:47.279304 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc8a7c-48a6-43c4-84f9-770e6f3fc98a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:47.288747 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddc8a7c-48a6-43c4-84f9-770e6f3fc98a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:47.291573 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddc8a7c-48a6-43c4-84f9-770e6f3fc98a service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:47.294079 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:47.300638 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 958/1937] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:42:47 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:42:55.210020 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e87f636-6930-4256-883e-5f6006dd9e86 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:55.273054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e87f636-6930-4256-883e-5f6006dd9e86 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:55.277626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e809b5-ef21-44aa-8fe7-6cf8087f33d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:55.278249 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e809b5-ef21-44aa-8fe7-6cf8087f33d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:55.278526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e809b5-ef21-44aa-8fe7-6cf8087f33d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:55.278796 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e809b5-ef21-44aa-8fe7-6cf8087f33d2 admin admin] PATH_INFO: `/v3/users/8abe7d184c124110bb39780bc61b294d` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:55.279902 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e809b5-ef21-44aa-8fe7-6cf8087f33d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8abe7d184c124110bb39780bc61b294d)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:55.281933 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e809b5-ef21-44aa-8fe7-6cf8087f33d2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:55.443937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-17e809b5-ef21-44aa-8fe7-6cf8087f33d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8abe7d184c124110bb39780bc61b294d'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:55.444671 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-17e809b5-ef21-44aa-8fe7-6cf8087f33d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8abe7d184c124110bb39780bc61b294d'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:55.445366 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-17e809b5-ef21-44aa-8fe7-6cf8087f33d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8abe7d184c124110bb39780bc61b294d'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:55.452509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-17e809b5-ef21-44aa-8fe7-6cf8087f33d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8abe7d184c124110bb39780bc61b294d'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:55.561571 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-17e809b5-ef21-44aa-8fe7-6cf8087f33d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8abe7d184c124110bb39780bc61b294d'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:55.582148 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 980/1938] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:42:55 2022] DELETE /identity/v3/users/8abe7d184c124110bb39780bc61b294d => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:55.586956 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad282a6-76cc-4c62-99e3-bab523002c26 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:55.586956 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad282a6-76cc-4c62-99e3-bab523002c26 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:55.587402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad282a6-76cc-4c62-99e3-bab523002c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:55.606439 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-3ad282a6-76cc-4c62-99e3-bab523002c26 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:55.607700 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-3ad282a6-76cc-4c62-99e3-bab523002c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:55.673681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 959/1939] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:55.683542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e809b5-ef21-44aa-8fe7-6cf8087f33d2 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:55.684934 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-17e809b5-ef21-44aa-8fe7-6cf8087f33d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:55.692663 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:55.744935 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e809b5-ef21-44aa-8fe7-6cf8087f33d2 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:55.748800 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91af5a-cc16-4515-b717-3ade73c7b7a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:55.749730 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91af5a-cc16-4515-b717-3ade73c7b7a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:55.750184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91af5a-cc16-4515-b717-3ade73c7b7a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:55.751276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91af5a-cc16-4515-b717-3ade73c7b7a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:55.761721 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb91af5a-cc16-4515-b717-3ade73c7b7a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:55.763230 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb91af5a-cc16-4515-b717-3ade73c7b7a9 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:55.774233 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 981/1940] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:42:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:42:56.216004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad282a6-76cc-4c62-99e3-bab523002c26 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:56.216004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-3ad282a6-76cc-4c62-99e3-bab523002c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:56.253184 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad282a6-76cc-4c62-99e3-bab523002c26 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:56.280685 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f0dd3-1d58-4e36-b2f9-b90a9b5d57c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:56.281608 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f0dd3-1d58-4e36-b2f9-b90a9b5d57c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:56.282087 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f0dd3-1d58-4e36-b2f9-b90a9b5d57c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:56.283045 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f0dd3-1d58-4e36-b2f9-b90a9b5d57c6 admin admin] PATH_INFO: `/v3/projects/7b33311164464811836d22bdf73b800a` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:56.284715 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f0dd3-1d58-4e36-b2f9-b90a9b5d57c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b33311164464811836d22bdf73b800a)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:56.286720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f0dd3-1d58-4e36-b2f9-b90a9b5d57c6 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:56.414504 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f91f0dd3-1d58-4e36-b2f9-b90a9b5d57c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b33311164464811836d22bdf73b800a', 'request_id': 'req-f91f0dd3-1d58-4e36-b2f9-b90a9b5d57c6'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:56.422086 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f91f0dd3-1d58-4e36-b2f9-b90a9b5d57c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b33311164464811836d22bdf73b800a', 'request_id': 'req-f91f0dd3-1d58-4e36-b2f9-b90a9b5d57c6'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:56.428875 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f91f0dd3-1d58-4e36-b2f9-b90a9b5d57c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b33311164464811836d22bdf73b800a', 'request_id': 'req-f91f0dd3-1d58-4e36-b2f9-b90a9b5d57c6'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:56.499649 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f91f0dd3-1d58-4e36-b2f9-b90a9b5d57c6 admin admin] The token cache is being invalidate because project 7b33311164464811836d22bdf73b800a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:42:56.500039 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f91f0dd3-1d58-4e36-b2f9-b90a9b5d57c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:56.500781 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f91f0dd3-1d58-4e36-b2f9-b90a9b5d57c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:56.505591 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 960/1941] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:42:56 2022] DELETE /identity/v3/projects/7b33311164464811836d22bdf73b800a => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:56.507330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91af5a-cc16-4515-b717-3ade73c7b7a9 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:56.521496 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91af5a-cc16-4515-b717-3ade73c7b7a9 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:56.582403 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf2d0b-3f5a-4dd5-9c73-f7be587f3536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:56.583241 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf2d0b-3f5a-4dd5-9c73-f7be587f3536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:56.583345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf2d0b-3f5a-4dd5-9c73-f7be587f3536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:56.583562 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf2d0b-3f5a-4dd5-9c73-f7be587f3536 admin admin] PATH_INFO: `/v3/users/9299911c4f1747c4a44134a4aac1f01c` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:56.584769 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf2d0b-3f5a-4dd5-9c73-f7be587f3536 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9299911c4f1747c4a44134a4aac1f01c)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:56.587410 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf2d0b-3f5a-4dd5-9c73-f7be587f3536 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:56.832861 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c5bf2d0b-3f5a-4dd5-9c73-f7be587f3536 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9299911c4f1747c4a44134a4aac1f01c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:56.833518 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c5bf2d0b-3f5a-4dd5-9c73-f7be587f3536 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9299911c4f1747c4a44134a4aac1f01c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:56.834207 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c5bf2d0b-3f5a-4dd5-9c73-f7be587f3536 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9299911c4f1747c4a44134a4aac1f01c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:56.841899 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c5bf2d0b-3f5a-4dd5-9c73-f7be587f3536 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9299911c4f1747c4a44134a4aac1f01c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:56.933063 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c5bf2d0b-3f5a-4dd5-9c73-f7be587f3536 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9299911c4f1747c4a44134a4aac1f01c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:56.952023 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 982/1942] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:42:56 2022] DELETE /identity/v3/users/9299911c4f1747c4a44134a4aac1f01c => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:57.363088 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f0dd3-1d58-4e36-b2f9-b90a9b5d57c6 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:57.365071 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f91f0dd3-1d58-4e36-b2f9-b90a9b5d57c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:57.380220 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f0dd3-1d58-4e36-b2f9-b90a9b5d57c6 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:57.450317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1dca05-db63-48d0-9b6f-4e2b5a25686e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:57.451053 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1dca05-db63-48d0-9b6f-4e2b5a25686e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:57.451404 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1dca05-db63-48d0-9b6f-4e2b5a25686e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:57.451750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1dca05-db63-48d0-9b6f-4e2b5a25686e admin admin] PATH_INFO: `/v3/projects/2f735573634743bca32ff83b049e299a` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:57.452949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1dca05-db63-48d0-9b6f-4e2b5a25686e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f735573634743bca32ff83b049e299a)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:57.455530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1dca05-db63-48d0-9b6f-4e2b5a25686e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:57.482292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:57.575359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1b1dca05-db63-48d0-9b6f-4e2b5a25686e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f735573634743bca32ff83b049e299a', 'request_id': 'req-1b1dca05-db63-48d0-9b6f-4e2b5a25686e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:57.581700 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1b1dca05-db63-48d0-9b6f-4e2b5a25686e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f735573634743bca32ff83b049e299a', 'request_id': 'req-1b1dca05-db63-48d0-9b6f-4e2b5a25686e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:57.587166 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1b1dca05-db63-48d0-9b6f-4e2b5a25686e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f735573634743bca32ff83b049e299a', 'request_id': 'req-1b1dca05-db63-48d0-9b6f-4e2b5a25686e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:57.652193 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1b1dca05-db63-48d0-9b6f-4e2b5a25686e admin admin] The token cache is being invalidate because project 2f735573634743bca32ff83b049e299a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:42:57.652396 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1b1dca05-db63-48d0-9b6f-4e2b5a25686e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:57.652927 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1b1dca05-db63-48d0-9b6f-4e2b5a25686e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:57.654466 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 961/1943] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:42:57 2022] DELETE /identity/v3/projects/2f735573634743bca32ff83b049e299a => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:57.659069 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf2d0b-3f5a-4dd5-9c73-f7be587f3536 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:57.662549 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c5bf2d0b-3f5a-4dd5-9c73-f7be587f3536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:57.679793 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf2d0b-3f5a-4dd5-9c73-f7be587f3536 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:57.709496 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:57.747995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e282ee0-2f06-4d69-929d-9cdfb7e3c114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:57.748856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e282ee0-2f06-4d69-929d-9cdfb7e3c114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:57.749600 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e282ee0-2f06-4d69-929d-9cdfb7e3c114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:57.749600 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e282ee0-2f06-4d69-929d-9cdfb7e3c114 admin admin] PATH_INFO: `/v3/users/c3c1606bbf73426883b588131fef6ad4` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:57.750679 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e282ee0-2f06-4d69-929d-9cdfb7e3c114 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3c1606bbf73426883b588131fef6ad4)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:57.753406 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e282ee0-2f06-4d69-929d-9cdfb7e3c114 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:57.917817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1e282ee0-2f06-4d69-929d-9cdfb7e3c114 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3c1606bbf73426883b588131fef6ad4'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:57.918542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1e282ee0-2f06-4d69-929d-9cdfb7e3c114 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3c1606bbf73426883b588131fef6ad4'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:57.919155 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1e282ee0-2f06-4d69-929d-9cdfb7e3c114 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3c1606bbf73426883b588131fef6ad4'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:57.926622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1e282ee0-2f06-4d69-929d-9cdfb7e3c114 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3c1606bbf73426883b588131fef6ad4'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:58.005519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1e282ee0-2f06-4d69-929d-9cdfb7e3c114 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3c1606bbf73426883b588131fef6ad4'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:58.025629 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 983/1944] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:42:57 2022] DELETE /identity/v3/users/c3c1606bbf73426883b588131fef6ad4 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:58.432612 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1dca05-db63-48d0-9b6f-4e2b5a25686e admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:58.436599 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1b1dca05-db63-48d0-9b6f-4e2b5a25686e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:58.454336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1dca05-db63-48d0-9b6f-4e2b5a25686e admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:58.524048 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7bfcf-c9e2-49b8-bbc7-465fea3c0e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:58.525735 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7bfcf-c9e2-49b8-bbc7-465fea3c0e8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:58.526067 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7bfcf-c9e2-49b8-bbc7-465fea3c0e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:58.526414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7bfcf-c9e2-49b8-bbc7-465fea3c0e8a admin admin] PATH_INFO: `/v3/projects/a29562a2853c41afa064bda60e1f72be` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:58.527637 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7bfcf-c9e2-49b8-bbc7-465fea3c0e8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a29562a2853c41afa064bda60e1f72be)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:58.529616 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7bfcf-c9e2-49b8-bbc7-465fea3c0e8a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:58.661732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-9ea7bfcf-c9e2-49b8-bbc7-465fea3c0e8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a29562a2853c41afa064bda60e1f72be', 'request_id': 'req-9ea7bfcf-c9e2-49b8-bbc7-465fea3c0e8a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:58.668068 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-9ea7bfcf-c9e2-49b8-bbc7-465fea3c0e8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a29562a2853c41afa064bda60e1f72be', 'request_id': 'req-9ea7bfcf-c9e2-49b8-bbc7-465fea3c0e8a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:58.674188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-9ea7bfcf-c9e2-49b8-bbc7-465fea3c0e8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a29562a2853c41afa064bda60e1f72be', 'request_id': 'req-9ea7bfcf-c9e2-49b8-bbc7-465fea3c0e8a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:58.795307 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-9ea7bfcf-c9e2-49b8-bbc7-465fea3c0e8a admin admin] The token cache is being invalidate because project a29562a2853c41afa064bda60e1f72be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:42:58.797555 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-9ea7bfcf-c9e2-49b8-bbc7-465fea3c0e8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:58.797555 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-9ea7bfcf-c9e2-49b8-bbc7-465fea3c0e8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:42:58.798232 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 962/1945] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:42:58 2022] DELETE /identity/v3/projects/a29562a2853c41afa064bda60e1f72be => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:58.803909 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24052ffd-c811-4cb6-8979-98e7230442d7 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:58.804454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24052ffd-c811-4cb6-8979-98e7230442d7 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:58.804571 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24052ffd-c811-4cb6-8979-98e7230442d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:58.828180 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-24052ffd-c811-4cb6-8979-98e7230442d7 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:58.828374 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-24052ffd-c811-4cb6-8979-98e7230442d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:58.899616 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 984/1946] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:58.908463 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e56adb-996b-48f7-ad86-750c484a620d None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:58.908859 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e56adb-996b-48f7-ad86-750c484a620d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:58.909235 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e56adb-996b-48f7-ad86-750c484a620d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:58.931908 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-57e56adb-996b-48f7-ad86-750c484a620d None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:58.932108 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-57e56adb-996b-48f7-ad86-750c484a620d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:58.946150 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 963/1947] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:42:58.948834 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24052ffd-c811-4cb6-8979-98e7230442d7 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:58.964117 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24052ffd-c811-4cb6-8979-98e7230442d7 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:58.967960 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4094e-29ea-4f8b-bc39-2b325fbb3c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:58.968693 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4094e-29ea-4f8b-bc39-2b325fbb3c38 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:58.969032 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4094e-29ea-4f8b-bc39-2b325fbb3c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:58.969389 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4094e-29ea-4f8b-bc39-2b325fbb3c38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:58.970480 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4094e-29ea-4f8b-bc39-2b325fbb3c38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:42:58.970885 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4094e-29ea-4f8b-bc39-2b325fbb3c38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:58.972590 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4094e-29ea-4f8b-bc39-2b325fbb3c38 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:58.986861 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 985/1948] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:42:58.991407 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e56adb-996b-48f7-ad86-750c484a620d None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:59.001368 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e56adb-996b-48f7-ad86-750c484a620d None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:59.006130 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c10416-c3fe-45bb-9028-3de0ec416afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:59.006741 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c10416-c3fe-45bb-9028-3de0ec416afc admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:59.007018 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c10416-c3fe-45bb-9028-3de0ec416afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:59.007289 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c10416-c3fe-45bb-9028-3de0ec416afc admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:59.007912 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c10416-c3fe-45bb-9028-3de0ec416afc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:59.018809 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c10416-c3fe-45bb-9028-3de0ec416afc admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:59.109231 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 964/1949] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:42:58 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:42:59.110658 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4094e-29ea-4f8b-bc39-2b325fbb3c38 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:59.120633 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4094e-29ea-4f8b-bc39-2b325fbb3c38 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:59.165560 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:59.193676 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7bc98a-3687-471d-90d5-393de920e16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:59.194301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7bc98a-3687-471d-90d5-393de920e16e admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:59.194581 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7bc98a-3687-471d-90d5-393de920e16e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:59.194853 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7bc98a-3687-471d-90d5-393de920e16e admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:59.195498 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7bc98a-3687-471d-90d5-393de920e16e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:59.197681 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7bc98a-3687-471d-90d5-393de920e16e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:59.299603 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 986/1950] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:42:59 2022] POST /identity/v3/users => generated 292 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:42:59.303773 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c10416-c3fe-45bb-9028-3de0ec416afc admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:59.315647 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c10416-c3fe-45bb-9028-3de0ec416afc admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:59.320562 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a36c4-e816-481c-b606-889316e7ef4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:59.321295 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a36c4-e816-481c-b606-889316e7ef4f admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:59.326495 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a36c4-e816-481c-b606-889316e7ef4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:59.326495 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a36c4-e816-481c-b606-889316e7ef4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:59.326495 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a36c4-e816-481c-b606-889316e7ef4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:59.326495 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a36c4-e816-481c-b606-889316e7ef4f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:59.335123 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 965/1951] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:42:59.339211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7bc98a-3687-471d-90d5-393de920e16e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:59.350819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7bc98a-3687-471d-90d5-393de920e16e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:59.361074 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c38c0-73cd-417d-9ff6-613e7ce47b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:59.371800 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c38c0-73cd-417d-9ff6-613e7ce47b43 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:59.371800 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c38c0-73cd-417d-9ff6-613e7ce47b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:59.371800 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c38c0-73cd-417d-9ff6-613e7ce47b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:59.371800 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c38c0-73cd-417d-9ff6-613e7ce47b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:59.371800 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c38c0-73cd-417d-9ff6-613e7ce47b43 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:59.378835 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 987/1952] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:42:59.382794 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a36c4-e816-481c-b606-889316e7ef4f admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:59.392308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:42:59.398832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a36c4-e816-481c-b606-889316e7ef4f admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:59.404243 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a778159-0b5b-4546-9ca3-c466fd218a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:59.405075 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a778159-0b5b-4546-9ca3-c466fd218a5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:59.405574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a778159-0b5b-4546-9ca3-c466fd218a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:59.405996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a778159-0b5b-4546-9ca3-c466fd218a5a admin admin] PATH_INFO: `/v3/projects/5b7a13e9df1a40c4b96ca5d8e3488976/users/27861a3107ea4c5daeda9ac9509d802f/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:59.426955 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a778159-0b5b-4546-9ca3-c466fd218a5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b7a13e9df1a40c4b96ca5d8e3488976, user_id=27861a3107ea4c5daeda9ac9509d802f, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:59.442753 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a778159-0b5b-4546-9ca3-c466fd218a5a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:59.516668 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 966/1953] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:42:59 2022] PUT /identity/v3/projects/5b7a13e9df1a40c4b96ca5d8e3488976/users/27861a3107ea4c5daeda9ac9509d802f/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:42:59.527352 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c968c0-49c5-43dd-bd66-d62c1d93fb00 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:59.527646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c968c0-49c5-43dd-bd66-d62c1d93fb00 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:59.527907 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c968c0-49c5-43dd-bd66-d62c1d93fb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:59.560414 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-48c968c0-49c5-43dd-bd66-d62c1d93fb00 None None] MFA Rules not processed for user `27861a3107ea4c5daeda9ac9509d802f`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:42:59.561666 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-48c968c0-49c5-43dd-bd66-d62c1d93fb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:59.654527 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 988/1954] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:42:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:42:59.666187 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a778159-0b5b-4546-9ca3-c466fd218a5a admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:42:59.667908 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8a778159-0b5b-4546-9ca3-c466fd218a5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:42:59.732342 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a778159-0b5b-4546-9ca3-c466fd218a5a admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:42:59.736722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad04950-a74d-4009-9051-ebc317e9f4c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:42:59.737370 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad04950-a74d-4009-9051-ebc317e9f4c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:42:59.737657 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad04950-a74d-4009-9051-ebc317e9f4c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:42:59.737934 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad04950-a74d-4009-9051-ebc317e9f4c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:42:59.746289 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad04950-a74d-4009-9051-ebc317e9f4c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:42:59.747716 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad04950-a74d-4009-9051-ebc317e9f4c2 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:42:59.757972 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 967/1955] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:42:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:43:00.185200 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c968c0-49c5-43dd-bd66-d62c1d93fb00 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:00.186790 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-48c968c0-49c5-43dd-bd66-d62c1d93fb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:00.265668 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c968c0-49c5-43dd-bd66-d62c1d93fb00 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:00.270351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570eab6-5049-4f66-b67f-485e3f624bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:00.270984 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570eab6-5049-4f66-b67f-485e3f624bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:00.271261 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570eab6-5049-4f66-b67f-485e3f624bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:00.271530 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570eab6-5049-4f66-b67f-485e3f624bfb admin admin] PATH_INFO: `/v3/users/489aa74b7ab34d5789416739b8751473` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:00.272636 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4570eab6-5049-4f66-b67f-485e3f624bfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=489aa74b7ab34d5789416739b8751473)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:00.274601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4570eab6-5049-4f66-b67f-485e3f624bfb admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:00.477755 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:00.480283 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-4570eab6-5049-4f66-b67f-485e3f624bfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '489aa74b7ab34d5789416739b8751473'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:00.481384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-4570eab6-5049-4f66-b67f-485e3f624bfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '489aa74b7ab34d5789416739b8751473'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:00.481992 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-4570eab6-5049-4f66-b67f-485e3f624bfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '489aa74b7ab34d5789416739b8751473'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:00.489241 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-4570eab6-5049-4f66-b67f-485e3f624bfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '489aa74b7ab34d5789416739b8751473'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:00.607734 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-4570eab6-5049-4f66-b67f-485e3f624bfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '489aa74b7ab34d5789416739b8751473'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:00.647236 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 989/1956] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:43:00 2022] DELETE /identity/v3/users/489aa74b7ab34d5789416739b8751473 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:00.906172 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad04950-a74d-4009-9051-ebc317e9f4c2 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:00.920890 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad04950-a74d-4009-9051-ebc317e9f4c2 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:00.993381 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75dbc9-18f1-4fa4-a702-f5998ebf0e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:00.994407 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75dbc9-18f1-4fa4-a702-f5998ebf0e46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:00.995009 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75dbc9-18f1-4fa4-a702-f5998ebf0e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:00.998036 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75dbc9-18f1-4fa4-a702-f5998ebf0e46 admin admin] PATH_INFO: `/v3/projects/cee46083a26948a9a4aba2068f5c816c` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:00.998036 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75dbc9-18f1-4fa4-a702-f5998ebf0e46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cee46083a26948a9a4aba2068f5c816c)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:00.999710 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75dbc9-18f1-4fa4-a702-f5998ebf0e46 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:01.127579 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5b75dbc9-18f1-4fa4-a702-f5998ebf0e46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cee46083a26948a9a4aba2068f5c816c', 'request_id': 'req-5b75dbc9-18f1-4fa4-a702-f5998ebf0e46'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:01.137191 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5b75dbc9-18f1-4fa4-a702-f5998ebf0e46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cee46083a26948a9a4aba2068f5c816c', 'request_id': 'req-5b75dbc9-18f1-4fa4-a702-f5998ebf0e46'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:01.145031 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5b75dbc9-18f1-4fa4-a702-f5998ebf0e46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cee46083a26948a9a4aba2068f5c816c', 'request_id': 'req-5b75dbc9-18f1-4fa4-a702-f5998ebf0e46'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:01.253521 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5b75dbc9-18f1-4fa4-a702-f5998ebf0e46 admin admin] The token cache is being invalidate because project cee46083a26948a9a4aba2068f5c816c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:43:01.253732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5b75dbc9-18f1-4fa4-a702-f5998ebf0e46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:01.254352 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5b75dbc9-18f1-4fa4-a702-f5998ebf0e46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:01.255903 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 968/1957] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:43:00 2022] DELETE /identity/v3/projects/cee46083a26948a9a4aba2068f5c816c => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:01.260856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98475500-44e3-4508-9234-1d3d755b144a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:01.261361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98475500-44e3-4508-9234-1d3d755b144a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:01.261469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98475500-44e3-4508-9234-1d3d755b144a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:01.281662 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-98475500-44e3-4508-9234-1d3d755b144a None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:01.283250 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-98475500-44e3-4508-9234-1d3d755b144a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:01.356575 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 990/1958] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:01.366135 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51dc07c-b0b2-4f38-94f8-ef093eebd6b1 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:01.366910 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51dc07c-b0b2-4f38-94f8-ef093eebd6b1 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:01.367629 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51dc07c-b0b2-4f38-94f8-ef093eebd6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:01.390555 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-d51dc07c-b0b2-4f38-94f8-ef093eebd6b1 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:01.391787 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d51dc07c-b0b2-4f38-94f8-ef093eebd6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:01.403738 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 969/1959] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:01.408055 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98475500-44e3-4508-9234-1d3d755b144a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:01.420095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98475500-44e3-4508-9234-1d3d755b144a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:01.425825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f2d7a-c33b-4cbf-91cf-8141b2542aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:01.426447 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f2d7a-c33b-4cbf-91cf-8141b2542aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:01.426719 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f2d7a-c33b-4cbf-91cf-8141b2542aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:01.426988 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f2d7a-c33b-4cbf-91cf-8141b2542aa7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:01.427855 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f2d7a-c33b-4cbf-91cf-8141b2542aa7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:43:01.428236 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f2d7a-c33b-4cbf-91cf-8141b2542aa7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:01.429985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f2d7a-c33b-4cbf-91cf-8141b2542aa7 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:01.446971 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 991/1960] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:43:01.448239 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51dc07c-b0b2-4f38-94f8-ef093eebd6b1 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:01.455232 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:01.459676 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51dc07c-b0b2-4f38-94f8-ef093eebd6b1 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:01.465515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97c984-3a62-426d-9df9-b84071a9348a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:01.466455 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97c984-3a62-426d-9df9-b84071a9348a admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:01.467134 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97c984-3a62-426d-9df9-b84071a9348a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:01.467415 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97c984-3a62-426d-9df9-b84071a9348a admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:01.468131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97c984-3a62-426d-9df9-b84071a9348a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:01.471323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97c984-3a62-426d-9df9-b84071a9348a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:01.562778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 970/1961] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:43:01 2022] POST /identity/v3/projects => generated 363 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:43:01.567141 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f2d7a-c33b-4cbf-91cf-8141b2542aa7 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:01.578822 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f2d7a-c33b-4cbf-91cf-8141b2542aa7 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:01.648940 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8a44d-7618-44dd-bba4-7692343fb417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:01.649602 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8a44d-7618-44dd-bba4-7692343fb417 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:01.649889 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8a44d-7618-44dd-bba4-7692343fb417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:01.650164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8a44d-7618-44dd-bba4-7692343fb417 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:01.650806 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8a44d-7618-44dd-bba4-7692343fb417 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:01.652735 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8a44d-7618-44dd-bba4-7692343fb417 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:01.751547 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 992/1962] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:43:01 2022] POST /identity/v3/users => generated 282 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:43:01.761068 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97c984-3a62-426d-9df9-b84071a9348a admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:01.772393 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97c984-3a62-426d-9df9-b84071a9348a admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:01.777565 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad21bca-a7c1-48c7-9763-8c140e1447ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:01.778176 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad21bca-a7c1-48c7-9763-8c140e1447ce admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:01.778455 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad21bca-a7c1-48c7-9763-8c140e1447ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:01.778750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad21bca-a7c1-48c7-9763-8c140e1447ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:01.779398 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad21bca-a7c1-48c7-9763-8c140e1447ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:01.780986 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad21bca-a7c1-48c7-9763-8c140e1447ce admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:01.792058 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 971/1963] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:43:01.796218 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8a44d-7618-44dd-bba4-7692343fb417 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:01.807517 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8a44d-7618-44dd-bba4-7692343fb417 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:01.812179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921fab45-6adc-4e9a-b9cb-6e461019397a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:01.816278 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921fab45-6adc-4e9a-b9cb-6e461019397a admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:01.816278 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921fab45-6adc-4e9a-b9cb-6e461019397a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:01.816278 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921fab45-6adc-4e9a-b9cb-6e461019397a admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:01.816278 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921fab45-6adc-4e9a-b9cb-6e461019397a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:01.816278 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921fab45-6adc-4e9a-b9cb-6e461019397a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:01.826601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 993/1964] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:43:01.831747 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad21bca-a7c1-48c7-9763-8c140e1447ce admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:01.842562 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad21bca-a7c1-48c7-9763-8c140e1447ce admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:01.847926 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34428b47-0392-4fdd-b747-f4104d5f434d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:01.848586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34428b47-0392-4fdd-b747-f4104d5f434d admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:01.848873 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34428b47-0392-4fdd-b747-f4104d5f434d admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:01.849461 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34428b47-0392-4fdd-b747-f4104d5f434d admin admin] PATH_INFO: `/v3/projects/d1ac974a8b6e497bb2ffe57becb498fc/users/a740efef03d641f78170b778fc2d4e75/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:01.864708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34428b47-0392-4fdd-b747-f4104d5f434d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1ac974a8b6e497bb2ffe57becb498fc, user_id=a740efef03d641f78170b778fc2d4e75, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:01.867433 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34428b47-0392-4fdd-b747-f4104d5f434d admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:01.957728 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 972/1965] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:43:01 2022] PUT /identity/v3/projects/d1ac974a8b6e497bb2ffe57becb498fc/users/a740efef03d641f78170b778fc2d4e75/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:43:01.969116 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82b812-a603-440f-811a-2bb62c2ce1bb None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:01.969437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82b812-a603-440f-811a-2bb62c2ce1bb None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:01.970450 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82b812-a603-440f-811a-2bb62c2ce1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:02.002245 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ec82b812-a603-440f-811a-2bb62c2ce1bb None None] MFA Rules not processed for user `a740efef03d641f78170b778fc2d4e75`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:02.003450 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ec82b812-a603-440f-811a-2bb62c2ce1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:02.016224 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:02.069228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 994/1966] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:02.073700 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34428b47-0392-4fdd-b747-f4104d5f434d admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:02.136618 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34428b47-0392-4fdd-b747-f4104d5f434d admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:02.141336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d816202-0ed0-4229-92ff-873daa9ae987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:02.142665 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d816202-0ed0-4229-92ff-873daa9ae987 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:02.143028 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d816202-0ed0-4229-92ff-873daa9ae987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:02.143304 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d816202-0ed0-4229-92ff-873daa9ae987 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:02.143924 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d816202-0ed0-4229-92ff-873daa9ae987 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:02.145852 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d816202-0ed0-4229-92ff-873daa9ae987 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:02.234026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 973/1967] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:43:02 2022] POST /identity/v3/projects => generated 363 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:43:02.239511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82b812-a603-440f-811a-2bb62c2ce1bb None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:02.251985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82b812-a603-440f-811a-2bb62c2ce1bb None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:02.314688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ad244-10e8-456b-9b2f-1cf89777d6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:02.315406 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ad244-10e8-456b-9b2f-1cf89777d6e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:02.315950 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ad244-10e8-456b-9b2f-1cf89777d6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:02.316226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ad244-10e8-456b-9b2f-1cf89777d6e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:02.317028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ad244-10e8-456b-9b2f-1cf89777d6e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:02.318968 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ad244-10e8-456b-9b2f-1cf89777d6e0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:02.424325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 995/1968] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:43:02 2022] POST /identity/v3/users => generated 288 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:43:02.426046 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d816202-0ed0-4229-92ff-873daa9ae987 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:02.436413 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d816202-0ed0-4229-92ff-873daa9ae987 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:02.441668 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397de420-24d9-40ed-8934-d1c6998302b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:02.442159 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397de420-24d9-40ed-8934-d1c6998302b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:02.446120 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397de420-24d9-40ed-8934-d1c6998302b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:02.446120 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397de420-24d9-40ed-8934-d1c6998302b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:02.446120 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397de420-24d9-40ed-8934-d1c6998302b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:02.446120 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397de420-24d9-40ed-8934-d1c6998302b2 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:02.455808 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 974/1969] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:43:02.460116 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ad244-10e8-456b-9b2f-1cf89777d6e0 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:02.470916 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ad244-10e8-456b-9b2f-1cf89777d6e0 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:02.475805 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b0c8c-463e-4e83-bfac-dee2efd50eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:02.476447 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b0c8c-463e-4e83-bfac-dee2efd50eb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:02.476723 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b0c8c-463e-4e83-bfac-dee2efd50eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:02.476991 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b0c8c-463e-4e83-bfac-dee2efd50eb1 admin admin] PATH_INFO: `/v3/projects/71f284fa51bc479398633c0297176010/users/dec69e63114f4afcacc3dd7409b27260/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:02.492181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9b0c8c-463e-4e83-bfac-dee2efd50eb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71f284fa51bc479398633c0297176010, user_id=dec69e63114f4afcacc3dd7409b27260, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:02.494889 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9b0c8c-463e-4e83-bfac-dee2efd50eb1 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:02.591337 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 996/1970] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:43:02 2022] PUT /identity/v3/projects/71f284fa51bc479398633c0297176010/users/dec69e63114f4afcacc3dd7409b27260/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:02.592182 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397de420-24d9-40ed-8934-d1c6998302b2 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:02.603095 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397de420-24d9-40ed-8934-d1c6998302b2 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:02.660135 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903507c-8cae-4dde-9b6f-b88f6143f25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:02.660742 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903507c-8cae-4dde-9b6f-b88f6143f25e admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:02.661021 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903507c-8cae-4dde-9b6f-b88f6143f25e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:02.661322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903507c-8cae-4dde-9b6f-b88f6143f25e admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:02.661958 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903507c-8cae-4dde-9b6f-b88f6143f25e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:02.663525 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903507c-8cae-4dde-9b6f-b88f6143f25e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:02.673883 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 975/1971] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:43:02.677995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b0c8c-463e-4e83-bfac-dee2efd50eb1 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:02.689014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b0c8c-463e-4e83-bfac-dee2efd50eb1 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:02.693680 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234d965-23bc-4533-b29b-4907098421c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:02.694321 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234d965-23bc-4533-b29b-4907098421c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:02.694600 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234d965-23bc-4533-b29b-4907098421c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:02.694868 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234d965-23bc-4533-b29b-4907098421c4 admin admin] PATH_INFO: `/v3/projects/71f284fa51bc479398633c0297176010/users/dec69e63114f4afcacc3dd7409b27260/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:02.696745 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234d965-23bc-4533-b29b-4907098421c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71f284fa51bc479398633c0297176010, user_id=dec69e63114f4afcacc3dd7409b27260, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:02.699381 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234d965-23bc-4533-b29b-4907098421c4 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:02.709484 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 997/1972] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:43:02 2022] PUT /identity/v3/projects/71f284fa51bc479398633c0297176010/users/dec69e63114f4afcacc3dd7409b27260/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:02.721087 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4c6f0-33ec-4983-b356-7fc7917d6ee2 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:02.721414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4c6f0-33ec-4983-b356-7fc7917d6ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:02.721688 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4c6f0-33ec-4983-b356-7fc7917d6ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:02.749093 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-0db4c6f0-33ec-4983-b356-7fc7917d6ee2 None None] MFA Rules not processed for user `dec69e63114f4afcacc3dd7409b27260`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:02.750304 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0db4c6f0-33ec-4983-b356-7fc7917d6ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:02.755913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:02.817435 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 976/1973] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:02 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:02.828871 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234d965-23bc-4533-b29b-4907098421c4 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:02.830641 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a234d965-23bc-4533-b29b-4907098421c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:02.886305 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234d965-23bc-4533-b29b-4907098421c4 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:02.889865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c698f-fb39-49e4-a3e2-0642bff865ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:02.890456 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c698f-fb39-49e4-a3e2-0642bff865ff service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:02.890731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c698f-fb39-49e4-a3e2-0642bff865ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:02.890996 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c698f-fb39-49e4-a3e2-0642bff865ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:02.899595 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c698f-fb39-49e4-a3e2-0642bff865ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:02.900997 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c698f-fb39-49e4-a3e2-0642bff865ff service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:02.910100 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 998/1974] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:43:02 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:43:03.784899 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4c6f0-33ec-4983-b356-7fc7917d6ee2 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:03.846534 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4c6f0-33ec-4983-b356-7fc7917d6ee2 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:03.851320 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca9e3e-1ed4-47a2-ab9f-7e836d603677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:03.851949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca9e3e-1ed4-47a2-ab9f-7e836d603677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:03.852232 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca9e3e-1ed4-47a2-ab9f-7e836d603677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:03.852510 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca9e3e-1ed4-47a2-ab9f-7e836d603677 admin admin] PATH_INFO: `/v3/users/a740efef03d641f78170b778fc2d4e75` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:03.853656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ca9e3e-1ed4-47a2-ab9f-7e836d603677 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a740efef03d641f78170b778fc2d4e75)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:03.855558 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ca9e3e-1ed4-47a2-ab9f-7e836d603677 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:04.039722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-49ca9e3e-1ed4-47a2-ab9f-7e836d603677 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a740efef03d641f78170b778fc2d4e75'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:04.040416 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-49ca9e3e-1ed4-47a2-ab9f-7e836d603677 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a740efef03d641f78170b778fc2d4e75'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:04.041017 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-49ca9e3e-1ed4-47a2-ab9f-7e836d603677 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a740efef03d641f78170b778fc2d4e75'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:04.048671 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-49ca9e3e-1ed4-47a2-ab9f-7e836d603677 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a740efef03d641f78170b778fc2d4e75'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:04.109291 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-49ca9e3e-1ed4-47a2-ab9f-7e836d603677 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a740efef03d641f78170b778fc2d4e75'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:04.127372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 977/1975] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:43:03 2022] DELETE /identity/v3/users/a740efef03d641f78170b778fc2d4e75 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:04.131897 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acb16a-317a-4a33-865c-06db8442d6cc None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:04.132400 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acb16a-317a-4a33-865c-06db8442d6cc None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:04.132500 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acb16a-317a-4a33-865c-06db8442d6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:04.153180 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c5acb16a-317a-4a33-865c-06db8442d6cc None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:04.154425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c5acb16a-317a-4a33-865c-06db8442d6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:04.160222 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:04.211991 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 999/1976] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:04.222354 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca9e3e-1ed4-47a2-ab9f-7e836d603677 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:04.223818 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-49ca9e3e-1ed4-47a2-ab9f-7e836d603677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:04.252734 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:04.285116 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca9e3e-1ed4-47a2-ab9f-7e836d603677 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:04.288642 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6068e069-5fbc-4530-a663-8c35a44a5b53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:04.289310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068e069-5fbc-4530-a663-8c35a44a5b53 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:04.293269 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068e069-5fbc-4530-a663-8c35a44a5b53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:04.293269 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068e069-5fbc-4530-a663-8c35a44a5b53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:04.298817 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6068e069-5fbc-4530-a663-8c35a44a5b53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:04.300202 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6068e069-5fbc-4530-a663-8c35a44a5b53 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:04.310169 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 978/1977] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:43:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:43:04.781476 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5acb16a-317a-4a33-865c-06db8442d6cc None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:04.783142 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c5acb16a-317a-4a33-865c-06db8442d6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:04.796745 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5acb16a-317a-4a33-865c-06db8442d6cc None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:04.804330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229665b-8a64-4c6d-80de-c40c0908266c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:04.804985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229665b-8a64-4c6d-80de-c40c0908266c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:04.805473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229665b-8a64-4c6d-80de-c40c0908266c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:04.805752 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229665b-8a64-4c6d-80de-c40c0908266c admin admin] PATH_INFO: `/v3/projects/d1ac974a8b6e497bb2ffe57becb498fc` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:04.806877 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b229665b-8a64-4c6d-80de-c40c0908266c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1ac974a8b6e497bb2ffe57becb498fc)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:04.809911 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b229665b-8a64-4c6d-80de-c40c0908266c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:04.911257 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b229665b-8a64-4c6d-80de-c40c0908266c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1ac974a8b6e497bb2ffe57becb498fc', 'request_id': 'req-b229665b-8a64-4c6d-80de-c40c0908266c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:04.995929 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b229665b-8a64-4c6d-80de-c40c0908266c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1ac974a8b6e497bb2ffe57becb498fc', 'request_id': 'req-b229665b-8a64-4c6d-80de-c40c0908266c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:05.003590 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b229665b-8a64-4c6d-80de-c40c0908266c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1ac974a8b6e497bb2ffe57becb498fc', 'request_id': 'req-b229665b-8a64-4c6d-80de-c40c0908266c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:05.014292 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b229665b-8a64-4c6d-80de-c40c0908266c admin admin] The token cache is being invalidate because project d1ac974a8b6e497bb2ffe57becb498fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:43:05.014539 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b229665b-8a64-4c6d-80de-c40c0908266c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:05.015164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b229665b-8a64-4c6d-80de-c40c0908266c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:05.016699 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1000/1978] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:43:04 2022] DELETE /identity/v3/projects/d1ac974a8b6e497bb2ffe57becb498fc => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:05.021168 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6068e069-5fbc-4530-a663-8c35a44a5b53 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:05.030761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6068e069-5fbc-4530-a663-8c35a44a5b53 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:05.101796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6ab0b-9a07-4666-af50-6cf2d816df4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:05.102668 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6ab0b-9a07-4666-af50-6cf2d816df4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:05.103121 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6ab0b-9a07-4666-af50-6cf2d816df4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:05.103395 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6ab0b-9a07-4666-af50-6cf2d816df4e admin admin] PATH_INFO: `/v3/users/dec69e63114f4afcacc3dd7409b27260` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:05.104674 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6ab0b-9a07-4666-af50-6cf2d816df4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=dec69e63114f4afcacc3dd7409b27260)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:05.106654 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6ab0b-9a07-4666-af50-6cf2d816df4e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:05.300358 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229665b-8a64-4c6d-80de-c40c0908266c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:05.301786 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b229665b-8a64-4c6d-80de-c40c0908266c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:05.314555 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229665b-8a64-4c6d-80de-c40c0908266c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:05.319247 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f051ab-dfd9-4f99-ac4e-35df9dd448a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:05.319871 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f051ab-dfd9-4f99-ac4e-35df9dd448a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:05.320149 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f051ab-dfd9-4f99-ac4e-35df9dd448a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:05.320418 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f051ab-dfd9-4f99-ac4e-35df9dd448a2 admin admin] PATH_INFO: `/v3/users/27861a3107ea4c5daeda9ac9509d802f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:05.321672 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f051ab-dfd9-4f99-ac4e-35df9dd448a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27861a3107ea4c5daeda9ac9509d802f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:05.323461 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f051ab-dfd9-4f99-ac4e-35df9dd448a2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:05.323841 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a7a6ab0b-9a07-4666-af50-6cf2d816df4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dec69e63114f4afcacc3dd7409b27260'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:05.324506 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a7a6ab0b-9a07-4666-af50-6cf2d816df4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dec69e63114f4afcacc3dd7409b27260'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:05.325218 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a7a6ab0b-9a07-4666-af50-6cf2d816df4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dec69e63114f4afcacc3dd7409b27260'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:05.332519 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a7a6ab0b-9a07-4666-af50-6cf2d816df4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dec69e63114f4afcacc3dd7409b27260'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:05.414052 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a7a6ab0b-9a07-4666-af50-6cf2d816df4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dec69e63114f4afcacc3dd7409b27260'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:05.433741 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 979/1979] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:43:05 2022] DELETE /identity/v3/users/dec69e63114f4afcacc3dd7409b27260 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:05.479641 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-00f051ab-dfd9-4f99-ac4e-35df9dd448a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27861a3107ea4c5daeda9ac9509d802f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:05.480008 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-00f051ab-dfd9-4f99-ac4e-35df9dd448a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27861a3107ea4c5daeda9ac9509d802f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:05.480648 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-00f051ab-dfd9-4f99-ac4e-35df9dd448a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27861a3107ea4c5daeda9ac9509d802f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:05.488533 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-00f051ab-dfd9-4f99-ac4e-35df9dd448a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27861a3107ea4c5daeda9ac9509d802f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:05.590303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-00f051ab-dfd9-4f99-ac4e-35df9dd448a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27861a3107ea4c5daeda9ac9509d802f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:05.610267 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1001/1980] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:43:05 2022] DELETE /identity/v3/users/27861a3107ea4c5daeda9ac9509d802f => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:43:05.614619 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae9863-b16b-4cd8-aca8-15f7c3a0d680 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:05.615019 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae9863-b16b-4cd8-aca8-15f7c3a0d680 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:05.615437 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae9863-b16b-4cd8-aca8-15f7c3a0d680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:05.636896 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-e2ae9863-b16b-4cd8-aca8-15f7c3a0d680 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:05.638296 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e2ae9863-b16b-4cd8-aca8-15f7c3a0d680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:05.684256 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:05.702188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 980/1981] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:43:05.712515 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f051ab-dfd9-4f99-ac4e-35df9dd448a2 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:05.714044 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-00f051ab-dfd9-4f99-ac4e-35df9dd448a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:05.732325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:05.780357 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f051ab-dfd9-4f99-ac4e-35df9dd448a2 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:05.784240 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c2522c-6fb2-41cc-b0ec-082d0b840e6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:05.784851 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2522c-6fb2-41cc-b0ec-082d0b840e6f service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:05.785222 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2522c-6fb2-41cc-b0ec-082d0b840e6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:05.785565 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2522c-6fb2-41cc-b0ec-082d0b840e6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:05.795924 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c2522c-6fb2-41cc-b0ec-082d0b840e6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:05.798993 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c2522c-6fb2-41cc-b0ec-082d0b840e6f service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:05.812967 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1002/1982] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:43:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:43:05.873665 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae9863-b16b-4cd8-aca8-15f7c3a0d680 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:05.877987 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e2ae9863-b16b-4cd8-aca8-15f7c3a0d680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:05.899906 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae9863-b16b-4cd8-aca8-15f7c3a0d680 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:05.908421 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8326d4-0a36-403c-aeab-a0f2f4cb1bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:05.909464 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8326d4-0a36-403c-aeab-a0f2f4cb1bed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:05.909842 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8326d4-0a36-403c-aeab-a0f2f4cb1bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:05.910213 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8326d4-0a36-403c-aeab-a0f2f4cb1bed admin admin] PATH_INFO: `/v3/projects/71f284fa51bc479398633c0297176010` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:05.913089 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8326d4-0a36-403c-aeab-a0f2f4cb1bed admin admin] RBAC: Authorizing `identity:delete_project(project_id=71f284fa51bc479398633c0297176010)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:05.928742 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8326d4-0a36-403c-aeab-a0f2f4cb1bed admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:06.042606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5b8326d4-0a36-403c-aeab-a0f2f4cb1bed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71f284fa51bc479398633c0297176010', 'request_id': 'req-5b8326d4-0a36-403c-aeab-a0f2f4cb1bed'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:06.049757 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5b8326d4-0a36-403c-aeab-a0f2f4cb1bed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71f284fa51bc479398633c0297176010', 'request_id': 'req-5b8326d4-0a36-403c-aeab-a0f2f4cb1bed'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:06.055453 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5b8326d4-0a36-403c-aeab-a0f2f4cb1bed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71f284fa51bc479398633c0297176010', 'request_id': 'req-5b8326d4-0a36-403c-aeab-a0f2f4cb1bed'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:06.135094 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5b8326d4-0a36-403c-aeab-a0f2f4cb1bed admin admin] The token cache is being invalidate because project 71f284fa51bc479398633c0297176010 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:43:06.145171 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5b8326d4-0a36-403c-aeab-a0f2f4cb1bed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:06.145171 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5b8326d4-0a36-403c-aeab-a0f2f4cb1bed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:06.145171 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 981/1983] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:43:05 2022] DELETE /identity/v3/projects/71f284fa51bc479398633c0297176010 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:06.145651 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03763789-b143-42d6-b43c-d4d3b39523d0 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:06.145651 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03763789-b143-42d6-b43c-d4d3b39523d0 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:06.145651 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03763789-b143-42d6-b43c-d4d3b39523d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:06.167237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-03763789-b143-42d6-b43c-d4d3b39523d0 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:06.169435 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-03763789-b143-42d6-b43c-d4d3b39523d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:06.249768 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1003/1984] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:06.258512 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b4643-b678-4c5f-9998-cabe5e968a62 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:06.258803 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b4643-b678-4c5f-9998-cabe5e968a62 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:06.259084 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b4643-b678-4c5f-9998-cabe5e968a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:06.279726 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-154b4643-b678-4c5f-9998-cabe5e968a62 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:06.280985 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-154b4643-b678-4c5f-9998-cabe5e968a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:06.289459 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 982/1985] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:06.293962 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03763789-b143-42d6-b43c-d4d3b39523d0 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:06.303833 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03763789-b143-42d6-b43c-d4d3b39523d0 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:06.308578 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac70886-d2cc-48c4-aa5a-3ef368aa4ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:06.309254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac70886-d2cc-48c4-aa5a-3ef368aa4ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:06.309525 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac70886-d2cc-48c4-aa5a-3ef368aa4ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:06.309794 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac70886-d2cc-48c4-aa5a-3ef368aa4ba4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:06.310543 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac70886-d2cc-48c4-aa5a-3ef368aa4ba4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:43:06.310847 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac70886-d2cc-48c4-aa5a-3ef368aa4ba4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:06.312454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac70886-d2cc-48c4-aa5a-3ef368aa4ba4 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:06.328191 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1004/1986] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:43:06.330834 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b4643-b678-4c5f-9998-cabe5e968a62 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:06.341109 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b4643-b678-4c5f-9998-cabe5e968a62 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:06.346571 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b434d4db-3f6a-43af-b354-1beee585e6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:06.347186 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434d4db-3f6a-43af-b354-1beee585e6cc admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:06.349546 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434d4db-3f6a-43af-b354-1beee585e6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:06.349546 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434d4db-3f6a-43af-b354-1beee585e6cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:06.349546 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b434d4db-3f6a-43af-b354-1beee585e6cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:06.350550 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b434d4db-3f6a-43af-b354-1beee585e6cc admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:06.381069 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac70886-d2cc-48c4-aa5a-3ef368aa4ba4 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:06.382865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fac70886-d2cc-48c4-aa5a-3ef368aa4ba4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:06.399458 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac70886-d2cc-48c4-aa5a-3ef368aa4ba4 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:06.405032 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e53643-f3c1-4683-8527-c852ac3a836c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:06.407142 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e53643-f3c1-4683-8527-c852ac3a836c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:06.407540 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e53643-f3c1-4683-8527-c852ac3a836c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:06.407748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e53643-f3c1-4683-8527-c852ac3a836c admin admin] PATH_INFO: `/v3/projects/5b7a13e9df1a40c4b96ca5d8e3488976` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:06.415979 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e53643-f3c1-4683-8527-c852ac3a836c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b7a13e9df1a40c4b96ca5d8e3488976)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:06.418208 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e53643-f3c1-4683-8527-c852ac3a836c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:06.436078 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 983/1987] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:43:06 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:43:06.441676 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b434d4db-3f6a-43af-b354-1beee585e6cc admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:06.513615 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b434d4db-3f6a-43af-b354-1beee585e6cc admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:06.519383 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4bec7-3f3d-48f6-8d49-d916d8d52b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:06.520386 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4bec7-3f3d-48f6-8d49-d916d8d52b88 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:06.520738 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4bec7-3f3d-48f6-8d49-d916d8d52b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:06.521085 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4bec7-3f3d-48f6-8d49-d916d8d52b88 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:06.521803 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab4bec7-3f3d-48f6-8d49-d916d8d52b88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:06.523728 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab4bec7-3f3d-48f6-8d49-d916d8d52b88 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:06.536492 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c1e53643-f3c1-4683-8527-c852ac3a836c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b7a13e9df1a40c4b96ca5d8e3488976', 'request_id': 'req-c1e53643-f3c1-4683-8527-c852ac3a836c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:06.607499 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c1e53643-f3c1-4683-8527-c852ac3a836c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b7a13e9df1a40c4b96ca5d8e3488976', 'request_id': 'req-c1e53643-f3c1-4683-8527-c852ac3a836c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:06.616441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 984/1988] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:43:06 2022] POST /identity/v3/users => generated 294 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:43:06.618653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4bec7-3f3d-48f6-8d49-d916d8d52b88 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:06.620562 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-aab4bec7-3f3d-48f6-8d49-d916d8d52b88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:06.620715 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c1e53643-f3c1-4683-8527-c852ac3a836c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b7a13e9df1a40c4b96ca5d8e3488976', 'request_id': 'req-c1e53643-f3c1-4683-8527-c852ac3a836c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:06.634828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c1e53643-f3c1-4683-8527-c852ac3a836c admin admin] The token cache is being invalidate because project 5b7a13e9df1a40c4b96ca5d8e3488976 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:43:06.635117 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c1e53643-f3c1-4683-8527-c852ac3a836c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:06.635821 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c1e53643-f3c1-4683-8527-c852ac3a836c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:06.636521 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4bec7-3f3d-48f6-8d49-d916d8d52b88 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:06.652570 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1005/1989] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:43:06 2022] DELETE /identity/v3/projects/5b7a13e9df1a40c4b96ca5d8e3488976 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:06.652570 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e53643-f3c1-4683-8527-c852ac3a836c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:06.652570 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c1e53643-f3c1-4683-8527-c852ac3a836c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:06.680971 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:06.690237 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:06.731706 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c938c-ac31-4e60-a467-2f0e3399ea68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:06.732499 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c938c-ac31-4e60-a467-2f0e3399ea68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:06.732857 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c938c-ac31-4e60-a467-2f0e3399ea68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:06.733229 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c938c-ac31-4e60-a467-2f0e3399ea68 admin admin] PATH_INFO: `/v3/users/4dd0fe71ad3440c6a733fcb2b0df1cc6` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:06.734575 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c938c-ac31-4e60-a467-2f0e3399ea68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dd0fe71ad3440c6a733fcb2b0df1cc6)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:06.736303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e53643-f3c1-4683-8527-c852ac3a836c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:06.737554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c938c-ac31-4e60-a467-2f0e3399ea68 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:06.741888 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d444b89-7d58-4162-b4a3-edc59d2badb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:06.742849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d444b89-7d58-4162-b4a3-edc59d2badb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:06.743208 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d444b89-7d58-4162-b4a3-edc59d2badb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:06.743544 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d444b89-7d58-4162-b4a3-edc59d2badb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:06.744274 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d444b89-7d58-4162-b4a3-edc59d2badb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:06.746051 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d444b89-7d58-4162-b4a3-edc59d2badb2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:06.757090 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1006/1990] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:43:06.761374 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d444b89-7d58-4162-b4a3-edc59d2badb2 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:06.773370 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d444b89-7d58-4162-b4a3-edc59d2badb2 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:06.778507 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1f48e-dd26-4681-a4e0-8d24e9ae2f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:06.779145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1f48e-dd26-4681-a4e0-8d24e9ae2f11 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:06.782797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1f48e-dd26-4681-a4e0-8d24e9ae2f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:06.782797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1f48e-dd26-4681-a4e0-8d24e9ae2f11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:06.782797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af1f48e-dd26-4681-a4e0-8d24e9ae2f11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:06.782797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af1f48e-dd26-4681-a4e0-8d24e9ae2f11 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:06.793435 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1007/1991] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:43:06.797568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1f48e-dd26-4681-a4e0-8d24e9ae2f11 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:06.808035 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1f48e-dd26-4681-a4e0-8d24e9ae2f11 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:06.812878 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c9d10-a0d5-4d0c-8478-a0156104a0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:06.814389 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c9d10-a0d5-4d0c-8478-a0156104a0d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:06.814707 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c9d10-a0d5-4d0c-8478-a0156104a0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:06.814985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c9d10-a0d5-4d0c-8478-a0156104a0d4 admin admin] PATH_INFO: `/v3/projects/a8178924940e411e8dcb943e2dbfc935/users/1e0eacd946e4466d82a82505d028bd43/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:06.834018 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c9d10-a0d5-4d0c-8478-a0156104a0d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8178924940e411e8dcb943e2dbfc935, user_id=1e0eacd946e4466d82a82505d028bd43, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:06.836877 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c9d10-a0d5-4d0c-8478-a0156104a0d4 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:06.911121 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1008/1992] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:43:06 2022] PUT /identity/v3/projects/a8178924940e411e8dcb943e2dbfc935/users/1e0eacd946e4466d82a82505d028bd43/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:43:06.913759 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c40c938c-ac31-4e60-a467-2f0e3399ea68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dd0fe71ad3440c6a733fcb2b0df1cc6'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:06.914465 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c40c938c-ac31-4e60-a467-2f0e3399ea68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dd0fe71ad3440c6a733fcb2b0df1cc6'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:06.915160 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c40c938c-ac31-4e60-a467-2f0e3399ea68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dd0fe71ad3440c6a733fcb2b0df1cc6'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:06.921313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ebce7e-4a68-415e-8b3b-8aba548ab412 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:06.921603 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ebce7e-4a68-415e-8b3b-8aba548ab412 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:06.921874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ebce7e-4a68-415e-8b3b-8aba548ab412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:06.922254 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c40c938c-ac31-4e60-a467-2f0e3399ea68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dd0fe71ad3440c6a733fcb2b0df1cc6'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:06.950274 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-96ebce7e-4a68-415e-8b3b-8aba548ab412 None None] MFA Rules not processed for user `1e0eacd946e4466d82a82505d028bd43`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:06.951483 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-96ebce7e-4a68-415e-8b3b-8aba548ab412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:07.006671 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c40c938c-ac31-4e60-a467-2f0e3399ea68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dd0fe71ad3440c6a733fcb2b0df1cc6'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:07.021882 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1009/1993] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:43:07.026187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ebce7e-4a68-415e-8b3b-8aba548ab412 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:07.027645 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-96ebce7e-4a68-415e-8b3b-8aba548ab412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:07.034616 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 985/1994] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:43:06 2022] DELETE /identity/v3/users/4dd0fe71ad3440c6a733fcb2b0df1cc6 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:07.039447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14c050-3d19-43fa-9996-973570b000db None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:07.039737 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14c050-3d19-43fa-9996-973570b000db None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:07.040014 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14c050-3d19-43fa-9996-973570b000db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:07.061043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-0b14c050-3d19-43fa-9996-973570b000db None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:07.062925 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0b14c050-3d19-43fa-9996-973570b000db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:07.103047 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ebce7e-4a68-415e-8b3b-8aba548ab412 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:07.123748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:07.132799 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 986/1995] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:07.143224 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b14c050-3d19-43fa-9996-973570b000db None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:07.145171 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0b14c050-3d19-43fa-9996-973570b000db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:07.171065 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8dabd4-c3df-41a9-bde0-59e76d41b3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:07.171690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8dabd4-c3df-41a9-bde0-59e76d41b3fe admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:07.172043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8dabd4-c3df-41a9-bde0-59e76d41b3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:07.172391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8dabd4-c3df-41a9-bde0-59e76d41b3fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:07.173741 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8dabd4-c3df-41a9-bde0-59e76d41b3fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:07.175654 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8dabd4-c3df-41a9-bde0-59e76d41b3fe admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:07.203244 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:07.207290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b14c050-3d19-43fa-9996-973570b000db None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:07.210827 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ef351-1375-47b7-8a25-1eab0747adf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:07.211431 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ef351-1375-47b7-8a25-1eab0747adf1 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:07.211799 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ef351-1375-47b7-8a25-1eab0747adf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:07.212034 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ef351-1375-47b7-8a25-1eab0747adf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:07.221498 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ef351-1375-47b7-8a25-1eab0747adf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:07.224078 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ef351-1375-47b7-8a25-1eab0747adf1 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:07.233020 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 987/1996] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:43:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:43:07.261208 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1010/1997] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:43:07 2022] POST /identity/v3/projects => generated 385 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:43:07.265455 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ef351-1375-47b7-8a25-1eab0747adf1 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:07.275474 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ef351-1375-47b7-8a25-1eab0747adf1 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:07.332948 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74012cb4-4c4c-421d-a128-5dd8862b714f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:07.333587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74012cb4-4c4c-421d-a128-5dd8862b714f admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:07.333866 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74012cb4-4c4c-421d-a128-5dd8862b714f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:07.334135 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74012cb4-4c4c-421d-a128-5dd8862b714f admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:07.334764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74012cb4-4c4c-421d-a128-5dd8862b714f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:07.336684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74012cb4-4c4c-421d-a128-5dd8862b714f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:07.427718 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 988/1998] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:43:07 2022] POST /identity/v3/users => generated 299 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:43:07.428981 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8dabd4-c3df-41a9-bde0-59e76d41b3fe admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:07.441778 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8dabd4-c3df-41a9-bde0-59e76d41b3fe admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:07.447044 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d1cb1-f4f6-44e7-a8d6-bfe1068de22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:07.447678 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d1cb1-f4f6-44e7-a8d6-bfe1068de22b admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:07.448014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d1cb1-f4f6-44e7-a8d6-bfe1068de22b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:07.448183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d1cb1-f4f6-44e7-a8d6-bfe1068de22b admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:07.448835 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d1cb1-f4f6-44e7-a8d6-bfe1068de22b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:07.450749 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d1cb1-f4f6-44e7-a8d6-bfe1068de22b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:07.462966 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1011/1999] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:43:07.467139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74012cb4-4c4c-421d-a128-5dd8862b714f admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:07.483385 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74012cb4-4c4c-421d-a128-5dd8862b714f admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:07.488868 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2950d04-e95e-4a3d-9b7e-8f140bffc330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:07.489558 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2950d04-e95e-4a3d-9b7e-8f140bffc330 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:07.489836 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2950d04-e95e-4a3d-9b7e-8f140bffc330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:07.490105 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2950d04-e95e-4a3d-9b7e-8f140bffc330 admin admin] PATH_INFO: `/v3/projects/3c0f47a2bc9e4c33988671a53c55540d/users/003daf0ea3ee4667ad9968281653cdf9/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:07.505743 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2950d04-e95e-4a3d-9b7e-8f140bffc330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c0f47a2bc9e4c33988671a53c55540d, user_id=003daf0ea3ee4667ad9968281653cdf9, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:07.508529 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2950d04-e95e-4a3d-9b7e-8f140bffc330 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:07.580075 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 989/2000] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:43:07 2022] PUT /identity/v3/projects/3c0f47a2bc9e4c33988671a53c55540d/users/003daf0ea3ee4667ad9968281653cdf9/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:43:07.584080 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d1cb1-f4f6-44e7-a8d6-bfe1068de22b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:07.594054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d1cb1-f4f6-44e7-a8d6-bfe1068de22b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:07.651531 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9405f89-e324-4ee2-ae14-5e1205ef0753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:07.652136 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9405f89-e324-4ee2-ae14-5e1205ef0753 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:07.652418 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9405f89-e324-4ee2-ae14-5e1205ef0753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:07.652768 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9405f89-e324-4ee2-ae14-5e1205ef0753 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:07.653359 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9405f89-e324-4ee2-ae14-5e1205ef0753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:07.655015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9405f89-e324-4ee2-ae14-5e1205ef0753 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:07.662273 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2950d04-e95e-4a3d-9b7e-8f140bffc330 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:07.664772 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a2950d04-e95e-4a3d-9b7e-8f140bffc330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:07.666757 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1012/2001] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:43:07.673835 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9405f89-e324-4ee2-ae14-5e1205ef0753 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:07.685397 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9405f89-e324-4ee2-ae14-5e1205ef0753 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:07.690765 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33f1fb-8810-4891-9524-6b0b6e513fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:07.691438 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33f1fb-8810-4891-9524-6b0b6e513fc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:07.691803 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33f1fb-8810-4891-9524-6b0b6e513fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:07.692099 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33f1fb-8810-4891-9524-6b0b6e513fc2 admin admin] PATH_INFO: `/v3/projects/3c0f47a2bc9e4c33988671a53c55540d/users/003daf0ea3ee4667ad9968281653cdf9/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:07.694161 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33f1fb-8810-4891-9524-6b0b6e513fc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c0f47a2bc9e4c33988671a53c55540d, user_id=003daf0ea3ee4667ad9968281653cdf9, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:07.695129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2950d04-e95e-4a3d-9b7e-8f140bffc330 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:07.696796 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f33f1fb-8810-4891-9524-6b0b6e513fc2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:07.700070 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e99edb-5f09-44c1-9876-42ace905ed55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:07.700805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e99edb-5f09-44c1-9876-42ace905ed55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:07.701189 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e99edb-5f09-44c1-9876-42ace905ed55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:07.701541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e99edb-5f09-44c1-9876-42ace905ed55 admin admin] PATH_INFO: `/v3/projects/876198200cb2444ca392b5cd7cbf9edb` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:07.702824 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e99edb-5f09-44c1-9876-42ace905ed55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=876198200cb2444ca392b5cd7cbf9edb)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:07.705408 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e99edb-5f09-44c1-9876-42ace905ed55 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:07.725226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1013/2002] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:43:07 2022] PUT /identity/v3/projects/3c0f47a2bc9e4c33988671a53c55540d/users/003daf0ea3ee4667ad9968281653cdf9/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:07.733443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420533b-07f9-4e5d-a086-5ff94c262f6d None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:07.733796 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420533b-07f9-4e5d-a086-5ff94c262f6d None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:07.734140 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420533b-07f9-4e5d-a086-5ff94c262f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:07.761990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-7420533b-07f9-4e5d-a086-5ff94c262f6d None None] MFA Rules not processed for user `003daf0ea3ee4667ad9968281653cdf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:07.763207 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7420533b-07f9-4e5d-a086-5ff94c262f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:07.828481 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:07.833097 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1014/2003] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:07 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:43:07.838278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-15e99edb-5f09-44c1-9876-42ace905ed55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '876198200cb2444ca392b5cd7cbf9edb', 'request_id': 'req-15e99edb-5f09-44c1-9876-42ace905ed55'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:07.844203 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420533b-07f9-4e5d-a086-5ff94c262f6d None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:07.845448 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-15e99edb-5f09-44c1-9876-42ace905ed55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '876198200cb2444ca392b5cd7cbf9edb', 'request_id': 'req-15e99edb-5f09-44c1-9876-42ace905ed55'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:07.846613 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7420533b-07f9-4e5d-a086-5ff94c262f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:07.851335 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-15e99edb-5f09-44c1-9876-42ace905ed55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '876198200cb2444ca392b5cd7cbf9edb', 'request_id': 'req-15e99edb-5f09-44c1-9876-42ace905ed55'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:07.900913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420533b-07f9-4e5d-a086-5ff94c262f6d None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:07.915921 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-15e99edb-5f09-44c1-9876-42ace905ed55 admin admin] The token cache is being invalidate because project 876198200cb2444ca392b5cd7cbf9edb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:43:07.916309 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-15e99edb-5f09-44c1-9876-42ace905ed55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:07.917045 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-15e99edb-5f09-44c1-9876-42ace905ed55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:07.918643 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 990/2004] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:43:07 2022] DELETE /identity/v3/projects/876198200cb2444ca392b5cd7cbf9edb => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:07.922509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e99edb-5f09-44c1-9876-42ace905ed55 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:07.924343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-15e99edb-5f09-44c1-9876-42ace905ed55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:07.961784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c418e-6345-4685-aaed-2e5b490df248 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:07.962411 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c418e-6345-4685-aaed-2e5b490df248 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:07.962756 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c418e-6345-4685-aaed-2e5b490df248 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:07.962952 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c418e-6345-4685-aaed-2e5b490df248 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:07.965128 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-db1c418e-6345-4685-aaed-2e5b490df248 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:07.976345 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:07.998486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e99edb-5f09-44c1-9876-42ace905ed55 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:08.003980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe7a96-d2a4-418e-9d65-9a2082b709a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:08.004611 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe7a96-d2a4-418e-9d65-9a2082b709a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:08.004889 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe7a96-d2a4-418e-9d65-9a2082b709a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:08.005180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe7a96-d2a4-418e-9d65-9a2082b709a7 admin admin] PATH_INFO: `/v3/users/548242188b114dddbbbad03d2699d2c9` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:08.006375 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fe7a96-d2a4-418e-9d65-9a2082b709a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=548242188b114dddbbbad03d2699d2c9)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:08.008268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fe7a96-d2a4-418e-9d65-9a2082b709a7 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:08.032047 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c418e-6345-4685-aaed-2e5b490df248 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:08.033994 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c418e-6345-4685-aaed-2e5b490df248 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:08.043545 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1015/2005] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:43:07 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:43:08.110150 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c418e-6345-4685-aaed-2e5b490df248 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:08.111802 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-db1c418e-6345-4685-aaed-2e5b490df248 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:08.125424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c418e-6345-4685-aaed-2e5b490df248 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:08.133033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b724b-bb10-414d-a03a-b663d87719b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:08.133033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b724b-bb10-414d-a03a-b663d87719b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:08.133033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b724b-bb10-414d-a03a-b663d87719b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:08.133033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b724b-bb10-414d-a03a-b663d87719b1 admin admin] PATH_INFO: `/v3/users/1e0eacd946e4466d82a82505d028bd43` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:08.133853 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b724b-bb10-414d-a03a-b663d87719b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e0eacd946e4466d82a82505d028bd43)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:08.135497 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b724b-bb10-414d-a03a-b663d87719b1 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:08.308704 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-13fe7a96-d2a4-418e-9d65-9a2082b709a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '548242188b114dddbbbad03d2699d2c9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:08.309443 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-13fe7a96-d2a4-418e-9d65-9a2082b709a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '548242188b114dddbbbad03d2699d2c9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:08.310068 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-13fe7a96-d2a4-418e-9d65-9a2082b709a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '548242188b114dddbbbad03d2699d2c9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:08.317665 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-13fe7a96-d2a4-418e-9d65-9a2082b709a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '548242188b114dddbbbad03d2699d2c9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:08.387137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-de6b724b-bb10-414d-a03a-b663d87719b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e0eacd946e4466d82a82505d028bd43'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:08.387790 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-de6b724b-bb10-414d-a03a-b663d87719b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e0eacd946e4466d82a82505d028bd43'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:08.388396 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-de6b724b-bb10-414d-a03a-b663d87719b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e0eacd946e4466d82a82505d028bd43'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:08.395276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-de6b724b-bb10-414d-a03a-b663d87719b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e0eacd946e4466d82a82505d028bd43'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:08.398783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:08.429250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-13fe7a96-d2a4-418e-9d65-9a2082b709a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '548242188b114dddbbbad03d2699d2c9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:08.447986 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 991/2006] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:43:07 2022] DELETE /identity/v3/users/548242188b114dddbbbad03d2699d2c9 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:08.503908 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-de6b724b-bb10-414d-a03a-b663d87719b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e0eacd946e4466d82a82505d028bd43'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:08.522145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1016/2007] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:43:08 2022] DELETE /identity/v3/users/1e0eacd946e4466d82a82505d028bd43 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:43:08.526833 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ac88e-c334-4504-995b-0987db0f1076 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:08.527215 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ac88e-c334-4504-995b-0987db0f1076 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:08.527558 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ac88e-c334-4504-995b-0987db0f1076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:08.547379 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-883ac88e-c334-4504-995b-0987db0f1076 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:08.549635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-883ac88e-c334-4504-995b-0987db0f1076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:08.609357 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 992/2008] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:08.619219 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b724b-bb10-414d-a03a-b663d87719b1 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:08.620626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-de6b724b-bb10-414d-a03a-b663d87719b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:08.686739 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b724b-bb10-414d-a03a-b663d87719b1 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:08.691481 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377bec87-dfec-416f-9c4a-4cadab262e34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:08.692105 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377bec87-dfec-416f-9c4a-4cadab262e34 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:08.692382 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377bec87-dfec-416f-9c4a-4cadab262e34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:08.692752 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377bec87-dfec-416f-9c4a-4cadab262e34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:08.703240 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377bec87-dfec-416f-9c4a-4cadab262e34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:08.705225 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377bec87-dfec-416f-9c4a-4cadab262e34 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:08.715432 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1017/2009] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:43:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:43:08.986013 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ac88e-c334-4504-995b-0987db0f1076 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:08.987706 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-883ac88e-c334-4504-995b-0987db0f1076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:09.005375 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ac88e-c334-4504-995b-0987db0f1076 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:09.012220 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d0a35-27e6-4fb9-8f53-ff40afd709ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:09.013007 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d0a35-27e6-4fb9-8f53-ff40afd709ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:09.013435 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d0a35-27e6-4fb9-8f53-ff40afd709ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:09.014166 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d0a35-27e6-4fb9-8f53-ff40afd709ae admin admin] PATH_INFO: `/v3/projects/2788422ffa024081bf5f6dea441fa2ce` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:09.015620 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d0a35-27e6-4fb9-8f53-ff40afd709ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=2788422ffa024081bf5f6dea441fa2ce)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:09.024092 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d0a35-27e6-4fb9-8f53-ff40afd709ae admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:09.126794 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-579d0a35-27e6-4fb9-8f53-ff40afd709ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2788422ffa024081bf5f6dea441fa2ce', 'request_id': 'req-579d0a35-27e6-4fb9-8f53-ff40afd709ae'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:09.133478 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-579d0a35-27e6-4fb9-8f53-ff40afd709ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2788422ffa024081bf5f6dea441fa2ce', 'request_id': 'req-579d0a35-27e6-4fb9-8f53-ff40afd709ae'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:09.134773 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:09.139795 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-579d0a35-27e6-4fb9-8f53-ff40afd709ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2788422ffa024081bf5f6dea441fa2ce', 'request_id': 'req-579d0a35-27e6-4fb9-8f53-ff40afd709ae'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:09.208519 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-579d0a35-27e6-4fb9-8f53-ff40afd709ae admin admin] The token cache is being invalidate because project 2788422ffa024081bf5f6dea441fa2ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:43:09.208939 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-579d0a35-27e6-4fb9-8f53-ff40afd709ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:09.209571 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-579d0a35-27e6-4fb9-8f53-ff40afd709ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:09.210021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377bec87-dfec-416f-9c4a-4cadab262e34 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:09.211477 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 993/2010] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:43:08 2022] DELETE /identity/v3/projects/2788422ffa024081bf5f6dea441fa2ce => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:09.215049 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-377bec87-dfec-416f-9c4a-4cadab262e34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:09.216619 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27963469-c6ca-46d9-a2b7-d3dbc4cc9292 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:09.217057 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27963469-c6ca-46d9-a2b7-d3dbc4cc9292 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:09.217482 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27963469-c6ca-46d9-a2b7-d3dbc4cc9292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:09.249322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377bec87-dfec-416f-9c4a-4cadab262e34 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:09.251688 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-27963469-c6ca-46d9-a2b7-d3dbc4cc9292 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:09.253210 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-27963469-c6ca-46d9-a2b7-d3dbc4cc9292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:09.310924 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92803ab-854f-408e-bb01-84441f33b91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:09.311555 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92803ab-854f-408e-bb01-84441f33b91f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:09.311843 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92803ab-854f-408e-bb01-84441f33b91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:09.312110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92803ab-854f-408e-bb01-84441f33b91f admin admin] PATH_INFO: `/v3/projects/a8178924940e411e8dcb943e2dbfc935` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:09.313895 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92803ab-854f-408e-bb01-84441f33b91f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8178924940e411e8dcb943e2dbfc935)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:09.318609 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 994/2011] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:09.319496 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92803ab-854f-408e-bb01-84441f33b91f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:09.325466 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f1869-e0cc-4237-95e3-955845039154 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:09.326389 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f1869-e0cc-4237-95e3-955845039154 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:09.326708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f1869-e0cc-4237-95e3-955845039154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:09.349942 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-2e1f1869-e0cc-4237-95e3-955845039154 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:09.351138 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2e1f1869-e0cc-4237-95e3-955845039154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:09.359388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 995/2012] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:09.364200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f1869-e0cc-4237-95e3-955845039154 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:09.374423 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f1869-e0cc-4237-95e3-955845039154 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:09.379971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef54440-f87a-459f-b4b6-60b7d27b0273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:09.380572 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef54440-f87a-459f-b4b6-60b7d27b0273 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:09.380847 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef54440-f87a-459f-b4b6-60b7d27b0273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:09.381136 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef54440-f87a-459f-b4b6-60b7d27b0273 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:09.381813 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef54440-f87a-459f-b4b6-60b7d27b0273 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:43:09.382118 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef54440-f87a-459f-b4b6-60b7d27b0273 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:09.383825 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef54440-f87a-459f-b4b6-60b7d27b0273 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:09.397671 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 996/2013] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:43:09.401825 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef54440-f87a-459f-b4b6-60b7d27b0273 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:09.416975 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef54440-f87a-459f-b4b6-60b7d27b0273 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:09.429525 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c92803ab-854f-408e-bb01-84441f33b91f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8178924940e411e8dcb943e2dbfc935', 'request_id': 'req-c92803ab-854f-408e-bb01-84441f33b91f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:09.484534 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a494a1-c60d-4e1b-aed4-6db6d8285b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:09.485215 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a494a1-c60d-4e1b-aed4-6db6d8285b62 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:09.485505 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a494a1-c60d-4e1b-aed4-6db6d8285b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:09.485778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a494a1-c60d-4e1b-aed4-6db6d8285b62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:09.486394 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a494a1-c60d-4e1b-aed4-6db6d8285b62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:09.488304 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a494a1-c60d-4e1b-aed4-6db6d8285b62 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:09.498609 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c92803ab-854f-408e-bb01-84441f33b91f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8178924940e411e8dcb943e2dbfc935', 'request_id': 'req-c92803ab-854f-408e-bb01-84441f33b91f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:09.505950 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c92803ab-854f-408e-bb01-84441f33b91f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8178924940e411e8dcb943e2dbfc935', 'request_id': 'req-c92803ab-854f-408e-bb01-84441f33b91f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:09.517388 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c92803ab-854f-408e-bb01-84441f33b91f admin admin] The token cache is being invalidate because project a8178924940e411e8dcb943e2dbfc935 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:43:09.517681 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c92803ab-854f-408e-bb01-84441f33b91f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:09.518419 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c92803ab-854f-408e-bb01-84441f33b91f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:09.520233 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1018/2014] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:43:09 2022] DELETE /identity/v3/projects/a8178924940e411e8dcb943e2dbfc935 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:43:09.529193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92803ab-854f-408e-bb01-84441f33b91f admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:09.530505 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c92803ab-854f-408e-bb01-84441f33b91f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:09.543475 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92803ab-854f-408e-bb01-84441f33b91f admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:09.548161 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c334733-40ee-4b05-bfd1-3f7270b73d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:09.548770 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c334733-40ee-4b05-bfd1-3f7270b73d26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:09.549047 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c334733-40ee-4b05-bfd1-3f7270b73d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:09.549347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c334733-40ee-4b05-bfd1-3f7270b73d26 admin admin] PATH_INFO: `/v3/users/003daf0ea3ee4667ad9968281653cdf9` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:09.550521 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c334733-40ee-4b05-bfd1-3f7270b73d26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=003daf0ea3ee4667ad9968281653cdf9)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:09.552423 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c334733-40ee-4b05-bfd1-3f7270b73d26 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:09.556917 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:09.575526 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 997/2015] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:43:09 2022] POST /identity/v3/projects => generated 371 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:43:09.580223 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a494a1-c60d-4e1b-aed4-6db6d8285b62 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:09.581848 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-27a494a1-c60d-4e1b-aed4-6db6d8285b62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:09.630301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:09.645279 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a494a1-c60d-4e1b-aed4-6db6d8285b62 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:09.650000 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24bdc3-8abf-4ef6-854c-cc5fefb00dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:09.650591 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24bdc3-8abf-4ef6-854c-cc5fefb00dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:09.650869 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24bdc3-8abf-4ef6-854c-cc5fefb00dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:09.651139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24bdc3-8abf-4ef6-854c-cc5fefb00dd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:09.651784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24bdc3-8abf-4ef6-854c-cc5fefb00dd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:09.653722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24bdc3-8abf-4ef6-854c-cc5fefb00dd2 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:09.739364 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 998/2016] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:43:09 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:43:09.743556 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24bdc3-8abf-4ef6-854c-cc5fefb00dd2 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:09.753825 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24bdc3-8abf-4ef6-854c-cc5fefb00dd2 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:09.758452 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053f98f-d6ac-4514-a41f-b50e923f262d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:09.759063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053f98f-d6ac-4514-a41f-b50e923f262d admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:09.762047 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053f98f-d6ac-4514-a41f-b50e923f262d admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:09.762047 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053f98f-d6ac-4514-a41f-b50e923f262d admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:09.762047 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2053f98f-d6ac-4514-a41f-b50e923f262d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:09.762047 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2053f98f-d6ac-4514-a41f-b50e923f262d admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:09.772039 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 999/2017] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:43:09.776036 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053f98f-d6ac-4514-a41f-b50e923f262d admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:09.785930 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053f98f-d6ac-4514-a41f-b50e923f262d admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:09.788122 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3c334733-40ee-4b05-bfd1-3f7270b73d26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '003daf0ea3ee4667ad9968281653cdf9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:09.788806 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3c334733-40ee-4b05-bfd1-3f7270b73d26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '003daf0ea3ee4667ad9968281653cdf9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:09.789417 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3c334733-40ee-4b05-bfd1-3f7270b73d26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '003daf0ea3ee4667ad9968281653cdf9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:09.790876 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc42ff-b2cc-4f6d-a007-9c18e969067e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:09.791516 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc42ff-b2cc-4f6d-a007-9c18e969067e admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:09.791782 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc42ff-b2cc-4f6d-a007-9c18e969067e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:09.792051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc42ff-b2cc-4f6d-a007-9c18e969067e admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:09.792678 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc42ff-b2cc-4f6d-a007-9c18e969067e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:09.794279 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc42ff-b2cc-4f6d-a007-9c18e969067e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:09.796594 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3c334733-40ee-4b05-bfd1-3f7270b73d26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '003daf0ea3ee4667ad9968281653cdf9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:09.806370 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1000/2018] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:43:09.810478 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc42ff-b2cc-4f6d-a007-9c18e969067e admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:09.812141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b6bc42ff-b2cc-4f6d-a007-9c18e969067e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:09.825382 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc42ff-b2cc-4f6d-a007-9c18e969067e admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:09.830596 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522adab0-66e3-4d01-a204-50d3b1249b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:09.831258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522adab0-66e3-4d01-a204-50d3b1249b19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:09.831532 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522adab0-66e3-4d01-a204-50d3b1249b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:09.831823 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522adab0-66e3-4d01-a204-50d3b1249b19 admin admin] PATH_INFO: `/v3/projects/66e29e74cbf54956b2e4f0bc69fae9b9/users/b8dcfa4d6e9e413e8c6ec70903170106/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:09.846220 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522adab0-66e3-4d01-a204-50d3b1249b19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66e29e74cbf54956b2e4f0bc69fae9b9, user_id=b8dcfa4d6e9e413e8c6ec70903170106, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:09.848847 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522adab0-66e3-4d01-a204-50d3b1249b19 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:09.858367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-3c334733-40ee-4b05-bfd1-3f7270b73d26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '003daf0ea3ee4667ad9968281653cdf9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:09.877653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1019/2019] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:43:09 2022] DELETE /identity/v3/users/003daf0ea3ee4667ad9968281653cdf9 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:43:09.905137 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1001/2020] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:43:09 2022] PUT /identity/v3/projects/66e29e74cbf54956b2e4f0bc69fae9b9/users/b8dcfa4d6e9e413e8c6ec70903170106/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:43:09.911926 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42bf295-28da-4761-ae48-19b921f8f897 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:09.912214 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42bf295-28da-4761-ae48-19b921f8f897 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:09.912488 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42bf295-28da-4761-ae48-19b921f8f897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:09.931775 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c42bf295-28da-4761-ae48-19b921f8f897 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:09.932957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c42bf295-28da-4761-ae48-19b921f8f897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:09.995738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1020/2021] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:10.005709 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522adab0-66e3-4d01-a204-50d3b1249b19 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:10.007399 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-522adab0-66e3-4d01-a204-50d3b1249b19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:10.064397 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522adab0-66e3-4d01-a204-50d3b1249b19 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:10.067984 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635945e-4d6f-45ea-9dd7-4122c9b85bc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:10.068591 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635945e-4d6f-45ea-9dd7-4122c9b85bc1 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:10.068867 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635945e-4d6f-45ea-9dd7-4122c9b85bc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:10.069134 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635945e-4d6f-45ea-9dd7-4122c9b85bc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:10.078787 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1635945e-4d6f-45ea-9dd7-4122c9b85bc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:10.080403 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1635945e-4d6f-45ea-9dd7-4122c9b85bc1 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:10.090457 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1002/2022] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:43:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:43:10.278886 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42bf295-28da-4761-ae48-19b921f8f897 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:10.279033 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635945e-4d6f-45ea-9dd7-4122c9b85bc1 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:10.279129 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c42bf295-28da-4761-ae48-19b921f8f897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:10.279276 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1635945e-4d6f-45ea-9dd7-4122c9b85bc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:10.292530 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42bf295-28da-4761-ae48-19b921f8f897 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:10.295139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635945e-4d6f-45ea-9dd7-4122c9b85bc1 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:10.299603 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc04eb7-2acb-4273-882a-6678aafe53d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:10.305000 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc04eb7-2acb-4273-882a-6678aafe53d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:10.305119 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9fc6b-035f-4b97-9b21-93df97ef08e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:10.305267 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc04eb7-2acb-4273-882a-6678aafe53d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:10.305267 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc04eb7-2acb-4273-882a-6678aafe53d5 admin admin] PATH_INFO: `/v3/projects/3c0f47a2bc9e4c33988671a53c55540d` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:10.305517 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9fc6b-035f-4b97-9b21-93df97ef08e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:10.305517 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9fc6b-035f-4b97-9b21-93df97ef08e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:10.305517 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9fc6b-035f-4b97-9b21-93df97ef08e9 admin admin] PATH_INFO: `/v3/users/5bc5329934e2438fb9adbe5aff0fb14b` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:10.305798 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc04eb7-2acb-4273-882a-6678aafe53d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c0f47a2bc9e4c33988671a53c55540d)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:10.305894 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb9fc6b-035f-4b97-9b21-93df97ef08e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bc5329934e2438fb9adbe5aff0fb14b)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:10.305980 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc04eb7-2acb-4273-882a-6678aafe53d5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:10.306063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb9fc6b-035f-4b97-9b21-93df97ef08e9 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:10.510328 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5cc04eb7-2acb-4273-882a-6678aafe53d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c0f47a2bc9e4c33988671a53c55540d', 'request_id': 'req-5cc04eb7-2acb-4273-882a-6678aafe53d5'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:10.526466 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ceb9fc6b-035f-4b97-9b21-93df97ef08e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bc5329934e2438fb9adbe5aff0fb14b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:10.527194 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ceb9fc6b-035f-4b97-9b21-93df97ef08e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bc5329934e2438fb9adbe5aff0fb14b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:10.527813 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ceb9fc6b-035f-4b97-9b21-93df97ef08e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bc5329934e2438fb9adbe5aff0fb14b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:10.536232 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ceb9fc6b-035f-4b97-9b21-93df97ef08e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bc5329934e2438fb9adbe5aff0fb14b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:10.579303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5cc04eb7-2acb-4273-882a-6678aafe53d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c0f47a2bc9e4c33988671a53c55540d', 'request_id': 'req-5cc04eb7-2acb-4273-882a-6678aafe53d5'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:10.585956 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5cc04eb7-2acb-4273-882a-6678aafe53d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c0f47a2bc9e4c33988671a53c55540d', 'request_id': 'req-5cc04eb7-2acb-4273-882a-6678aafe53d5'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:10.597484 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5cc04eb7-2acb-4273-882a-6678aafe53d5 admin admin] The token cache is being invalidate because project 3c0f47a2bc9e4c33988671a53c55540d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:43:10.597965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5cc04eb7-2acb-4273-882a-6678aafe53d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:10.598634 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5cc04eb7-2acb-4273-882a-6678aafe53d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:10.600422 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1021/2023] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:43:10 2022] DELETE /identity/v3/projects/3c0f47a2bc9e4c33988671a53c55540d => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:10.605139 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc32d3d-8b79-4879-9bdb-a0c8a114df85 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:10.605525 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc32d3d-8b79-4879-9bdb-a0c8a114df85 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:10.605801 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc32d3d-8b79-4879-9bdb-a0c8a114df85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:10.622934 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:10.628559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ddc32d3d-8b79-4879-9bdb-a0c8a114df85 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:10.629809 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ddc32d3d-8b79-4879-9bdb-a0c8a114df85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:10.664826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ceb9fc6b-035f-4b97-9b21-93df97ef08e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bc5329934e2438fb9adbe5aff0fb14b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:10.686091 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1003/2024] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:43:10 2022] DELETE /identity/v3/users/5bc5329934e2438fb9adbe5aff0fb14b => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:10.695371 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1de0fb-e638-4fd4-86a6-fd1b9ffd0979 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:10.695661 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1de0fb-e638-4fd4-86a6-fd1b9ffd0979 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:10.695938 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1de0fb-e638-4fd4-86a6-fd1b9ffd0979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:10.699476 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1022/2025] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:10.707782 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0579fce-be3e-47af-b2fe-5482113fcce0 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:10.710006 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0579fce-be3e-47af-b2fe-5482113fcce0 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:10.711271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0579fce-be3e-47af-b2fe-5482113fcce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:10.725065 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:10.732223 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-9a1de0fb-e638-4fd4-86a6-fd1b9ffd0979 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:10.733001 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-9a1de0fb-e638-4fd4-86a6-fd1b9ffd0979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:10.746710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-f0579fce-be3e-47af-b2fe-5482113fcce0 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:10.747934 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f0579fce-be3e-47af-b2fe-5482113fcce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:10.810610 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1004/2026] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:10.824192 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1de0fb-e638-4fd4-86a6-fd1b9ffd0979 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:10.824192 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-9a1de0fb-e638-4fd4-86a6-fd1b9ffd0979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:10.824482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1023/2027] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:10.825449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0579fce-be3e-47af-b2fe-5482113fcce0 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:10.835830 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0579fce-be3e-47af-b2fe-5482113fcce0 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:10.842450 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056fcd9-f6f7-450b-a6be-71f181092c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:10.843307 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056fcd9-f6f7-450b-a6be-71f181092c9b admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:10.843761 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056fcd9-f6f7-450b-a6be-71f181092c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:10.844028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056fcd9-f6f7-450b-a6be-71f181092c9b admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:10.845314 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056fcd9-f6f7-450b-a6be-71f181092c9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:43:10.845611 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056fcd9-f6f7-450b-a6be-71f181092c9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:10.847387 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056fcd9-f6f7-450b-a6be-71f181092c9b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:10.866016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1024/2028] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:43:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:43:10.867098 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056fcd9-f6f7-450b-a6be-71f181092c9b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:10.881355 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056fcd9-f6f7-450b-a6be-71f181092c9b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:10.889462 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807c153-0a3c-4f78-9fc1-a0f3b4252fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:10.890877 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807c153-0a3c-4f78-9fc1-a0f3b4252fc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:10.892061 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807c153-0a3c-4f78-9fc1-a0f3b4252fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:10.892632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807c153-0a3c-4f78-9fc1-a0f3b4252fc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:10.893887 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5807c153-0a3c-4f78-9fc1-a0f3b4252fc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:10.895911 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5807c153-0a3c-4f78-9fc1-a0f3b4252fc3 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:10.903765 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1de0fb-e638-4fd4-86a6-fd1b9ffd0979 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:10.910808 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193284b-b6e6-482c-94a6-bf1e481570c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:10.912609 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193284b-b6e6-482c-94a6-bf1e481570c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:10.913469 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193284b-b6e6-482c-94a6-bf1e481570c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:10.913928 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193284b-b6e6-482c-94a6-bf1e481570c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:10.929103 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193284b-b6e6-482c-94a6-bf1e481570c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:10.930622 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193284b-b6e6-482c-94a6-bf1e481570c0 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:10.941270 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1005/2029] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:43:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:43:11.015853 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1025/2030] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:43:10 2022] POST /identity/v3/projects => generated 367 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:43:11.020803 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193284b-b6e6-482c-94a6-bf1e481570c0 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:11.031168 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193284b-b6e6-482c-94a6-bf1e481570c0 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:11.093707 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addaec54-244d-4802-a920-e310105c01f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:11.094241 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addaec54-244d-4802-a920-e310105c01f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:11.094574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addaec54-244d-4802-a920-e310105c01f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:11.095374 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addaec54-244d-4802-a920-e310105c01f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:11.096129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addaec54-244d-4802-a920-e310105c01f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:11.098176 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addaec54-244d-4802-a920-e310105c01f1 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:11.184721 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1006/2031] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:43:11 2022] POST /identity/v3/users => generated 284 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:43:11.188828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807c153-0a3c-4f78-9fc1-a0f3b4252fc3 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:11.199954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807c153-0a3c-4f78-9fc1-a0f3b4252fc3 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:11.205576 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692a499-e4ba-45b7-8287-194913ca2b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:11.206323 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692a499-e4ba-45b7-8287-194913ca2b09 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:11.206606 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692a499-e4ba-45b7-8287-194913ca2b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:11.206882 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692a499-e4ba-45b7-8287-194913ca2b09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:11.207544 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6692a499-e4ba-45b7-8287-194913ca2b09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:11.209238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6692a499-e4ba-45b7-8287-194913ca2b09 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:11.220795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1026/2032] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:43:11.225341 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addaec54-244d-4802-a920-e310105c01f1 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:11.236136 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addaec54-244d-4802-a920-e310105c01f1 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:11.240922 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b216842-ea48-4229-8079-05c786386257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:11.241603 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b216842-ea48-4229-8079-05c786386257 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:11.241887 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b216842-ea48-4229-8079-05c786386257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:11.242160 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b216842-ea48-4229-8079-05c786386257 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:11.242816 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b216842-ea48-4229-8079-05c786386257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:11.244441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b216842-ea48-4229-8079-05c786386257 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:11.256204 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1007/2033] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:43:11.260356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692a499-e4ba-45b7-8287-194913ca2b09 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:11.271792 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692a499-e4ba-45b7-8287-194913ca2b09 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:11.276658 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a9697-2d47-4b15-bdaf-dadc718cb4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:11.277395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a9697-2d47-4b15-bdaf-dadc718cb4ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:11.277673 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a9697-2d47-4b15-bdaf-dadc718cb4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:11.277947 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a9697-2d47-4b15-bdaf-dadc718cb4ec admin admin] PATH_INFO: `/v3/projects/a60c451fe8914ae3bfcecb70dd3792c2/users/1b27217508f4433db2b04e863012f8e1/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:11.294369 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a9697-2d47-4b15-bdaf-dadc718cb4ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=a60c451fe8914ae3bfcecb70dd3792c2, user_id=1b27217508f4433db2b04e863012f8e1, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:11.297044 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a9697-2d47-4b15-bdaf-dadc718cb4ec admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:11.346361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1027/2034] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:43:11 2022] PUT /identity/v3/projects/a60c451fe8914ae3bfcecb70dd3792c2/users/1b27217508f4433db2b04e863012f8e1/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:11.357117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538af6ad-d314-4d37-851d-ab70b7671dc3 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:11.357439 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538af6ad-d314-4d37-851d-ab70b7671dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:11.357722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538af6ad-d314-4d37-851d-ab70b7671dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:11.390044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-538af6ad-d314-4d37-851d-ab70b7671dc3 None None] MFA Rules not processed for user `1b27217508f4433db2b04e863012f8e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:11.391285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-538af6ad-d314-4d37-851d-ab70b7671dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:11.410602 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:11.465235 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1008/2035] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:11.469715 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a9697-2d47-4b15-bdaf-dadc718cb4ec admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:11.514990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:11.536976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a9697-2d47-4b15-bdaf-dadc718cb4ec admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:11.542028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b39af00-47f2-4266-bb2c-bdcf554811aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:11.542711 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39af00-47f2-4266-bb2c-bdcf554811aa admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:11.543086 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39af00-47f2-4266-bb2c-bdcf554811aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:11.543357 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39af00-47f2-4266-bb2c-bdcf554811aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:11.543992 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b39af00-47f2-4266-bb2c-bdcf554811aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:11.546072 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b39af00-47f2-4266-bb2c-bdcf554811aa admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:11.643233 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1028/2036] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:43:11 2022] POST /identity/v3/projects => generated 365 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:43:11.647492 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538af6ad-d314-4d37-851d-ab70b7671dc3 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:11.658878 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538af6ad-d314-4d37-851d-ab70b7671dc3 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:11.716944 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b39af00-47f2-4266-bb2c-bdcf554811aa admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:11.716944 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1b39af00-47f2-4266-bb2c-bdcf554811aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:11.727519 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab3f8b-8090-4983-9b86-54c8d040b283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:11.728188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab3f8b-8090-4983-9b86-54c8d040b283 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:11.728507 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab3f8b-8090-4983-9b86-54c8d040b283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:11.728781 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab3f8b-8090-4983-9b86-54c8d040b283 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:11.729604 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab3f8b-8090-4983-9b86-54c8d040b283 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:11.731566 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab3f8b-8090-4983-9b86-54c8d040b283 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:11.810812 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b39af00-47f2-4266-bb2c-bdcf554811aa admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:11.819522 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bae1fa-6352-4ebd-92a1-9d62100ec974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:11.820229 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bae1fa-6352-4ebd-92a1-9d62100ec974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:11.820514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bae1fa-6352-4ebd-92a1-9d62100ec974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:11.821131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bae1fa-6352-4ebd-92a1-9d62100ec974 admin admin] PATH_INFO: `/v3/projects/968732d328d94879a5edb49a64ab3670` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:11.823798 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bae1fa-6352-4ebd-92a1-9d62100ec974 admin admin] RBAC: Authorizing `identity:delete_project(project_id=968732d328d94879a5edb49a64ab3670)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:11.824573 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bae1fa-6352-4ebd-92a1-9d62100ec974 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:11.828067 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1009/2037] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:43:11 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:43:11.830226 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fdfec-d5b0-4d0d-b3e5-dbd0e6c0b6f8 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:11.830808 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fdfec-d5b0-4d0d-b3e5-dbd0e6c0b6f8 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:11.831943 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fdfec-d5b0-4d0d-b3e5-dbd0e6c0b6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:11.876335 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-d73fdfec-d5b0-4d0d-b3e5-dbd0e6c0b6f8 None None] MFA Rules not processed for user `b8dcfa4d6e9e413e8c6ec70903170106`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:11.877705 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d73fdfec-d5b0-4d0d-b3e5-dbd0e6c0b6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:11.955630 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1010/2038] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:11.956644 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fdfec-d5b0-4d0d-b3e5-dbd0e6c0b6f8 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:11.968017 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fdfec-d5b0-4d0d-b3e5-dbd0e6c0b6f8 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:11.976909 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-81bae1fa-6352-4ebd-92a1-9d62100ec974 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '968732d328d94879a5edb49a64ab3670', 'request_id': 'req-81bae1fa-6352-4ebd-92a1-9d62100ec974'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:12.001816 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:12.027284 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c33c5-10b7-461f-ae88-0f44af69ecb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:12.027895 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c33c5-10b7-461f-ae88-0f44af69ecb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:12.028173 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c33c5-10b7-461f-ae88-0f44af69ecb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:12.028448 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c33c5-10b7-461f-ae88-0f44af69ecb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:12.029189 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c33c5-10b7-461f-ae88-0f44af69ecb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:12.030744 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c33c5-10b7-461f-ae88-0f44af69ecb1 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:12.041311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1011/2039] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:43:12.042156 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c33c5-10b7-461f-ae88-0f44af69ecb1 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:12.043786 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-191c33c5-10b7-461f-ae88-0f44af69ecb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:12.053303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-81bae1fa-6352-4ebd-92a1-9d62100ec974 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '968732d328d94879a5edb49a64ab3670', 'request_id': 'req-81bae1fa-6352-4ebd-92a1-9d62100ec974'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:12.056417 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c33c5-10b7-461f-ae88-0f44af69ecb1 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:12.060680 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-81bae1fa-6352-4ebd-92a1-9d62100ec974 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '968732d328d94879a5edb49a64ab3670', 'request_id': 'req-81bae1fa-6352-4ebd-92a1-9d62100ec974'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:12.061049 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a81f2-d58f-4aa0-a96d-caff9de491f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:12.061665 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a81f2-d58f-4aa0-a96d-caff9de491f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:12.061946 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a81f2-d58f-4aa0-a96d-caff9de491f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:12.062218 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a81f2-d58f-4aa0-a96d-caff9de491f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:12.062971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a81f2-d58f-4aa0-a96d-caff9de491f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:12.064901 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a81f2-d58f-4aa0-a96d-caff9de491f7 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:12.072267 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-81bae1fa-6352-4ebd-92a1-9d62100ec974 admin admin] The token cache is being invalidate because project 968732d328d94879a5edb49a64ab3670 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:43:12.072555 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-81bae1fa-6352-4ebd-92a1-9d62100ec974 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:12.073261 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-81bae1fa-6352-4ebd-92a1-9d62100ec974 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:12.074891 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1029/2040] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:43:11 2022] DELETE /identity/v3/projects/968732d328d94879a5edb49a64ab3670 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:12.075973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bae1fa-6352-4ebd-92a1-9d62100ec974 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:12.077315 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-81bae1fa-6352-4ebd-92a1-9d62100ec974 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:12.090233 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bae1fa-6352-4ebd-92a1-9d62100ec974 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:12.095323 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f699066-f384-4bc7-9c9e-ef6412df3a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:12.095984 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f699066-f384-4bc7-9c9e-ef6412df3a66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:12.096256 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f699066-f384-4bc7-9c9e-ef6412df3a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:12.096521 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f699066-f384-4bc7-9c9e-ef6412df3a66 admin admin] PATH_INFO: `/v3/projects/7f5970ea97a74badb44f77e733f0efe9/users/6d4097c02c4240db9f6aa6b4ce1f9a0c/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:12.113026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f699066-f384-4bc7-9c9e-ef6412df3a66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f5970ea97a74badb44f77e733f0efe9, user_id=6d4097c02c4240db9f6aa6b4ce1f9a0c, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:12.115759 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f699066-f384-4bc7-9c9e-ef6412df3a66 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:12.145345 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1012/2041] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:43:12 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:43:12.146589 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76186eaa-1c50-489a-85b5-27d4f409343c None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:12.146869 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76186eaa-1c50-489a-85b5-27d4f409343c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:12.147144 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76186eaa-1c50-489a-85b5-27d4f409343c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:12.166915 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-76186eaa-1c50-489a-85b5-27d4f409343c None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:12.168035 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-76186eaa-1c50-489a-85b5-27d4f409343c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:12.195388 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1030/2042] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:43:12 2022] PUT /identity/v3/projects/7f5970ea97a74badb44f77e733f0efe9/users/6d4097c02c4240db9f6aa6b4ce1f9a0c/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:12.196312 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f699066-f384-4bc7-9c9e-ef6412df3a66 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:12.199248 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7f699066-f384-4bc7-9c9e-ef6412df3a66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:12.230984 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1013/2043] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:12.238388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76186eaa-1c50-489a-85b5-27d4f409343c None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:12.239646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:12.241746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76186eaa-1c50-489a-85b5-27d4f409343c None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:12.272618 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f699066-f384-4bc7-9c9e-ef6412df3a66 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:12.278202 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1262e9a-8f58-4707-8eb2-f40ff9dce0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:12.278806 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1262e9a-8f58-4707-8eb2-f40ff9dce0d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:12.279081 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1262e9a-8f58-4707-8eb2-f40ff9dce0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:12.279344 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1262e9a-8f58-4707-8eb2-f40ff9dce0d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:12.279967 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1262e9a-8f58-4707-8eb2-f40ff9dce0d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:12.281905 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1262e9a-8f58-4707-8eb2-f40ff9dce0d5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:12.298858 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f85829-a886-4963-ab72-0d1e302f3018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:12.299464 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f85829-a886-4963-ab72-0d1e302f3018 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:12.299743 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f85829-a886-4963-ab72-0d1e302f3018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:12.300015 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f85829-a886-4963-ab72-0d1e302f3018 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:12.300651 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f85829-a886-4963-ab72-0d1e302f3018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:12.302261 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f85829-a886-4963-ab72-0d1e302f3018 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:12.311797 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1014/2044] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:43:12.312937 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268a0ed-55ed-4d88-8b92-fd5406ba49cd None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:12.313271 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268a0ed-55ed-4d88-8b92-fd5406ba49cd None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:12.313545 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268a0ed-55ed-4d88-8b92-fd5406ba49cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:12.332352 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-6268a0ed-55ed-4d88-8b92-fd5406ba49cd None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:12.333577 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6268a0ed-55ed-4d88-8b92-fd5406ba49cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:12.345940 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1015/2045] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:12.346731 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268a0ed-55ed-4d88-8b92-fd5406ba49cd None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:12.357552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268a0ed-55ed-4d88-8b92-fd5406ba49cd None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:12.362640 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3014b720-3663-461f-aa50-c30284d8b8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:12.363331 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014b720-3663-461f-aa50-c30284d8b8f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:12.363611 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014b720-3663-461f-aa50-c30284d8b8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:12.363884 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014b720-3663-461f-aa50-c30284d8b8f7 admin admin] PATH_INFO: `/v3/projects/7f5970ea97a74badb44f77e733f0efe9/users/6d4097c02c4240db9f6aa6b4ce1f9a0c/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:12.365851 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3014b720-3663-461f-aa50-c30284d8b8f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f5970ea97a74badb44f77e733f0efe9, user_id=6d4097c02c4240db9f6aa6b4ce1f9a0c, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:12.368674 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3014b720-3663-461f-aa50-c30284d8b8f7 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:12.379102 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1016/2046] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:43:12 2022] PUT /identity/v3/projects/7f5970ea97a74badb44f77e733f0efe9/users/6d4097c02c4240db9f6aa6b4ce1f9a0c/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:12.465567 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3014b720-3663-461f-aa50-c30284d8b8f7 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:12.465567 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:12.465567 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3014b720-3663-461f-aa50-c30284d8b8f7 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:12.465567 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d979b-04ed-40d9-9a50-c98d28de9c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:12.466635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d979b-04ed-40d9-9a50-c98d28de9c0e admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:12.466635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d979b-04ed-40d9-9a50-c98d28de9c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:12.466635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d979b-04ed-40d9-9a50-c98d28de9c0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:12.466635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d979b-04ed-40d9-9a50-c98d28de9c0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:43:12.466635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d979b-04ed-40d9-9a50-c98d28de9c0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:12.466635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d979b-04ed-40d9-9a50-c98d28de9c0e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:12.473697 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1017/2047] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:43:12.475256 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd80a8-94d2-4723-96eb-661611416823 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:12.475533 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd80a8-94d2-4723-96eb-661611416823 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:12.475806 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd80a8-94d2-4723-96eb-661611416823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:12.501760 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-90fd80a8-94d2-4723-96eb-661611416823 None None] MFA Rules not processed for user `6d4097c02c4240db9f6aa6b4ce1f9a0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:12.503089 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-90fd80a8-94d2-4723-96eb-661611416823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:12.517034 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1031/2048] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:43:12 2022] POST /identity/v3/users => generated 293 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:43:12.517895 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1262e9a-8f58-4707-8eb2-f40ff9dce0d5 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:12.527629 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1262e9a-8f58-4707-8eb2-f40ff9dce0d5 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:12.533081 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf4b8f-f20a-4f0e-9dea-4643d8467871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:12.533786 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf4b8f-f20a-4f0e-9dea-4643d8467871 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:12.534133 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf4b8f-f20a-4f0e-9dea-4643d8467871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:12.534406 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf4b8f-f20a-4f0e-9dea-4643d8467871 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:12.535026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cf4b8f-f20a-4f0e-9dea-4643d8467871 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:12.536920 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cf4b8f-f20a-4f0e-9dea-4643d8467871 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:12.573081 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1018/2049] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:12 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:12.574005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd80a8-94d2-4723-96eb-661611416823 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:12.584642 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd80a8-94d2-4723-96eb-661611416823 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:12.589306 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998ff0f-c264-4dcc-8ec7-b11267367649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:12.589900 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998ff0f-c264-4dcc-8ec7-b11267367649 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:12.590181 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998ff0f-c264-4dcc-8ec7-b11267367649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:12.590455 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998ff0f-c264-4dcc-8ec7-b11267367649 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:12.591106 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c998ff0f-c264-4dcc-8ec7-b11267367649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:12.592679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c998ff0f-c264-4dcc-8ec7-b11267367649 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:12.602954 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1019/2050] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:43:12.603734 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998ff0f-c264-4dcc-8ec7-b11267367649 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:12.605395 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c998ff0f-c264-4dcc-8ec7-b11267367649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:12.660399 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998ff0f-c264-4dcc-8ec7-b11267367649 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:12.664354 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c347937-ab14-45eb-850b-13e8045da061 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:12.664953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c347937-ab14-45eb-850b-13e8045da061 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:12.665264 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c347937-ab14-45eb-850b-13e8045da061 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:12.665569 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c347937-ab14-45eb-850b-13e8045da061 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:12.675231 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c347937-ab14-45eb-850b-13e8045da061 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:12.676627 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c347937-ab14-45eb-850b-13e8045da061 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:12.686361 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1020/2051] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:43:12 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:43:12.689136 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c347937-ab14-45eb-850b-13e8045da061 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:12.698222 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c347937-ab14-45eb-850b-13e8045da061 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:12.702733 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e733ee82-dc09-4005-b62c-2d68d8365a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:12.703406 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733ee82-dc09-4005-b62c-2d68d8365a72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:12.703687 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733ee82-dc09-4005-b62c-2d68d8365a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:12.704007 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733ee82-dc09-4005-b62c-2d68d8365a72 admin admin] PATH_INFO: `/v3/projects/f55cb27aea574dba90983629a5eb2423/users/27595f96542c46afb10a9ccd6c921f33/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:12.717792 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1032/2052] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:43:12 2022] POST /identity/v3/projects => generated 397 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:43:12.722169 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e733ee82-dc09-4005-b62c-2d68d8365a72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f55cb27aea574dba90983629a5eb2423, user_id=27595f96542c46afb10a9ccd6c921f33, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:12.722624 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf4b8f-f20a-4f0e-9dea-4643d8467871 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:12.725220 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e733ee82-dc09-4005-b62c-2d68d8365a72 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:12.784504 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf4b8f-f20a-4f0e-9dea-4643d8467871 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:12.789251 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473bea0-512e-4b3c-a1e5-1dbf73b3750e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:12.789849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473bea0-512e-4b3c-a1e5-1dbf73b3750e admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:12.790125 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473bea0-512e-4b3c-a1e5-1dbf73b3750e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:12.790389 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473bea0-512e-4b3c-a1e5-1dbf73b3750e admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:12.791008 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f473bea0-512e-4b3c-a1e5-1dbf73b3750e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:12.792928 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f473bea0-512e-4b3c-a1e5-1dbf73b3750e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:12.809171 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1021/2053] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:43:12 2022] PUT /identity/v3/projects/f55cb27aea574dba90983629a5eb2423/users/27595f96542c46afb10a9ccd6c921f33/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:12.814952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e733ee82-dc09-4005-b62c-2d68d8365a72 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:12.832341 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:12.875366 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e733ee82-dc09-4005-b62c-2d68d8365a72 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:12.880186 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0095746-a117-4a03-b515-aac62089e39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:12.883008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0095746-a117-4a03-b515-aac62089e39b admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:12.883008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0095746-a117-4a03-b515-aac62089e39b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:12.883008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0095746-a117-4a03-b515-aac62089e39b admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:12.883008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0095746-a117-4a03-b515-aac62089e39b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:12.883341 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1033/2054] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:43:12 2022] POST /identity/v3/users => generated 299 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:43:12.883662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0095746-a117-4a03-b515-aac62089e39b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:12.886554 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473bea0-512e-4b3c-a1e5-1dbf73b3750e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:12.895230 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1022/2055] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:43:12.896754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473bea0-512e-4b3c-a1e5-1dbf73b3750e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:12.898644 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0095746-a117-4a03-b515-aac62089e39b admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:12.902797 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a4e50-9714-416a-8810-beb153818c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:12.909288 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0095746-a117-4a03-b515-aac62089e39b admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:12.909701 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a4e50-9714-416a-8810-beb153818c0a admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:12.909991 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a4e50-9714-416a-8810-beb153818c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:12.910334 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a4e50-9714-416a-8810-beb153818c0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:12.910937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a4e50-9714-416a-8810-beb153818c0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:12.913414 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a4e50-9714-416a-8810-beb153818c0a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:12.914211 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e8b0c9-08d9-4a0b-8335-144395096177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:12.914858 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8b0c9-08d9-4a0b-8335-144395096177 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:12.915135 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8b0c9-08d9-4a0b-8335-144395096177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:12.915405 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8b0c9-08d9-4a0b-8335-144395096177 admin admin] PATH_INFO: `/v3/projects/f55cb27aea574dba90983629a5eb2423/users/27595f96542c46afb10a9ccd6c921f33/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:12.917362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e8b0c9-08d9-4a0b-8335-144395096177 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f55cb27aea574dba90983629a5eb2423, user_id=27595f96542c46afb10a9ccd6c921f33, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:12.919952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e8b0c9-08d9-4a0b-8335-144395096177 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:12.927497 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1034/2056] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:43:12.927497 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a4e50-9714-416a-8810-beb153818c0a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:12.936269 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1023/2057] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:43:12 2022] PUT /identity/v3/projects/f55cb27aea574dba90983629a5eb2423/users/27595f96542c46afb10a9ccd6c921f33/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:43:12.937546 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e8b0c9-08d9-4a0b-8335-144395096177 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:12.996306 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a4e50-9714-416a-8810-beb153818c0a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:13.013933 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e8b0c9-08d9-4a0b-8335-144395096177 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:13.019102 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf23e96-aa92-4eb5-a4ad-a154ee874a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:13.019725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf23e96-aa92-4eb5-a4ad-a154ee874a10 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:13.020006 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf23e96-aa92-4eb5-a4ad-a154ee874a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:13.020280 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf23e96-aa92-4eb5-a4ad-a154ee874a10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:13.020926 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf23e96-aa92-4eb5-a4ad-a154ee874a10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:13.022544 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf23e96-aa92-4eb5-a4ad-a154ee874a10 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:13.033764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1024/2058] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:43:13.037447 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:13.039298 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf23e96-aa92-4eb5-a4ad-a154ee874a10 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:13.050101 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf23e96-aa92-4eb5-a4ad-a154ee874a10 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:13.055401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b3152-e8df-4b8a-8499-a5417bbc39e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:13.056503 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b3152-e8df-4b8a-8499-a5417bbc39e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:13.056852 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b3152-e8df-4b8a-8499-a5417bbc39e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:13.057554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b3152-e8df-4b8a-8499-a5417bbc39e0 admin admin] PATH_INFO: `/v3/projects/f568ee82e48c4f729afa5aa56534803e/users/b111c4ab9ff0443b8e0e2334b415a1b7/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:13.069095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901599e-c514-43d8-be5c-eea008781181 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:13.070134 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901599e-c514-43d8-be5c-eea008781181 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:13.070659 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901599e-c514-43d8-be5c-eea008781181 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:13.071503 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901599e-c514-43d8-be5c-eea008781181 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:13.074509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b901599e-c514-43d8-be5c-eea008781181 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:13.075615 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b3152-e8df-4b8a-8499-a5417bbc39e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f568ee82e48c4f729afa5aa56534803e, user_id=b111c4ab9ff0443b8e0e2334b415a1b7, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:13.078507 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b3152-e8df-4b8a-8499-a5417bbc39e0 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:13.152858 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901599e-c514-43d8-be5c-eea008781181 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:13.154694 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901599e-c514-43d8-be5c-eea008781181 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:13.163699 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1035/2059] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:43:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:43:13.225096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1025/2060] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:43:13 2022] PUT /identity/v3/projects/f568ee82e48c4f729afa5aa56534803e/users/b111c4ab9ff0443b8e0e2334b415a1b7/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:43:13.236287 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957be9b6-3062-40a0-8e92-f845e5a2f42c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:13.236608 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957be9b6-3062-40a0-8e92-f845e5a2f42c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:13.236881 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957be9b6-3062-40a0-8e92-f845e5a2f42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:13.269243 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-957be9b6-3062-40a0-8e92-f845e5a2f42c None None] MFA Rules not processed for user `b111c4ab9ff0443b8e0e2334b415a1b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:13.269455 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-957be9b6-3062-40a0-8e92-f845e5a2f42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:13.344023 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1036/2061] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:13 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:13.356199 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b3152-e8df-4b8a-8499-a5417bbc39e0 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:13.359830 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c32b3152-e8df-4b8a-8499-a5417bbc39e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:13.422438 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b3152-e8df-4b8a-8499-a5417bbc39e0 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:13.427044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a77cc-1436-483b-8708-66aea1fe8dff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:13.427662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a77cc-1436-483b-8708-66aea1fe8dff service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:13.427936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a77cc-1436-483b-8708-66aea1fe8dff service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:13.428206 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a77cc-1436-483b-8708-66aea1fe8dff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:13.440329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a77cc-1436-483b-8708-66aea1fe8dff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:13.440329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a77cc-1436-483b-8708-66aea1fe8dff service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:13.447924 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1026/2062] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:43:13 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:43:14.338170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab1c1f-2ba2-4ca9-92f9-2d15e8f85467 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:14.338708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab1c1f-2ba2-4ca9-92f9-2d15e8f85467 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:14.338814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab1c1f-2ba2-4ca9-92f9-2d15e8f85467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:14.377867 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-28ab1c1f-2ba2-4ca9-92f9-2d15e8f85467 None None] MFA Rules not processed for user `27595f96542c46afb10a9ccd6c921f33`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:14.379108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-28ab1c1f-2ba2-4ca9-92f9-2d15e8f85467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:14.465488 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:14.469027 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1037/2063] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:14.481552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a77cc-1436-483b-8708-66aea1fe8dff service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:14.490465 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a77cc-1436-483b-8708-66aea1fe8dff service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:14.494426 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87e7cd-4c3a-4d87-a0e4-2831380a4997 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:14.495015 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87e7cd-4c3a-4d87-a0e4-2831380a4997 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:14.495297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87e7cd-4c3a-4d87-a0e4-2831380a4997 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:14.495569 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87e7cd-4c3a-4d87-a0e4-2831380a4997 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:14.497597 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6f87e7cd-4c3a-4d87-a0e4-2831380a4997 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:14.509428 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:14.558853 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87e7cd-4c3a-4d87-a0e4-2831380a4997 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:14.560231 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87e7cd-4c3a-4d87-a0e4-2831380a4997 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:14.569554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1027/2064] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:43:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:43:16.280679 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab1c1f-2ba2-4ca9-92f9-2d15e8f85467 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:16.290708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab1c1f-2ba2-4ca9-92f9-2d15e8f85467 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:16.294219 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6399854-6834-48df-894e-f372cd2e1918 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:16.294814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6399854-6834-48df-894e-f372cd2e1918 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:16.295091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6399854-6834-48df-894e-f372cd2e1918 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:16.295358 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6399854-6834-48df-894e-f372cd2e1918 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:16.304050 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6399854-6834-48df-894e-f372cd2e1918 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:16.305512 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6399854-6834-48df-894e-f372cd2e1918 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:16.314815 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1038/2065] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:43:16 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:43:19.924878 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87e7cd-4c3a-4d87-a0e4-2831380a4997 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:20.009195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87e7cd-4c3a-4d87-a0e4-2831380a4997 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:20.017917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2aae46-c401-468b-952c-4463ea124707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:20.018700 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2aae46-c401-468b-952c-4463ea124707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:20.019080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2aae46-c401-468b-952c-4463ea124707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:20.019477 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2aae46-c401-468b-952c-4463ea124707 admin admin] PATH_INFO: `/v3/users/1b27217508f4433db2b04e863012f8e1` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:20.020732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2aae46-c401-468b-952c-4463ea124707 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b27217508f4433db2b04e863012f8e1)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:20.022921 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2aae46-c401-468b-952c-4463ea124707 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:20.253928 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5e2aae46-c401-468b-952c-4463ea124707 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b27217508f4433db2b04e863012f8e1'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:20.253928 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5e2aae46-c401-468b-952c-4463ea124707 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b27217508f4433db2b04e863012f8e1'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:20.253928 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5e2aae46-c401-468b-952c-4463ea124707 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b27217508f4433db2b04e863012f8e1'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:20.257791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5e2aae46-c401-468b-952c-4463ea124707 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b27217508f4433db2b04e863012f8e1'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:20.355876 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5e2aae46-c401-468b-952c-4463ea124707 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b27217508f4433db2b04e863012f8e1'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:20.374325 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1028/2066] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:43:19 2022] DELETE /identity/v3/users/1b27217508f4433db2b04e863012f8e1 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:20.378815 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59075b4c-5c00-4199-a753-671308ce0914 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:20.379296 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59075b4c-5c00-4199-a753-671308ce0914 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:20.379389 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59075b4c-5c00-4199-a753-671308ce0914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:20.400530 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-59075b4c-5c00-4199-a753-671308ce0914 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:20.400530 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-59075b4c-5c00-4199-a753-671308ce0914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:20.503162 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1039/2067] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:20.503878 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2aae46-c401-468b-952c-4463ea124707 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:20.503878 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5e2aae46-c401-468b-952c-4463ea124707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:20.534343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:20.558747 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2aae46-c401-468b-952c-4463ea124707 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:20.562395 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c908d-a09b-4833-8512-d2fd2deb7aed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:20.563065 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c908d-a09b-4833-8512-d2fd2deb7aed service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:20.563414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c908d-a09b-4833-8512-d2fd2deb7aed service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:20.563693 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c908d-a09b-4833-8512-d2fd2deb7aed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:20.572921 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c908d-a09b-4833-8512-d2fd2deb7aed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:20.574382 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c908d-a09b-4833-8512-d2fd2deb7aed service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:20.584503 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1029/2068] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:43:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:43:20.946630 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59075b4c-5c00-4199-a753-671308ce0914 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:20.948906 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-59075b4c-5c00-4199-a753-671308ce0914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:20.968569 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59075b4c-5c00-4199-a753-671308ce0914 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:20.976970 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ab098-2dcc-4dfa-b798-90d80c0e817a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:20.978076 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ab098-2dcc-4dfa-b798-90d80c0e817a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:20.978566 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ab098-2dcc-4dfa-b798-90d80c0e817a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:20.978944 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ab098-2dcc-4dfa-b798-90d80c0e817a admin admin] PATH_INFO: `/v3/projects/a60c451fe8914ae3bfcecb70dd3792c2` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:20.980432 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ab098-2dcc-4dfa-b798-90d80c0e817a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a60c451fe8914ae3bfcecb70dd3792c2)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:20.983179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ab098-2dcc-4dfa-b798-90d80c0e817a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:21.121457 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-030ab098-2dcc-4dfa-b798-90d80c0e817a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a60c451fe8914ae3bfcecb70dd3792c2', 'request_id': 'req-030ab098-2dcc-4dfa-b798-90d80c0e817a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:21.218944 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-030ab098-2dcc-4dfa-b798-90d80c0e817a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a60c451fe8914ae3bfcecb70dd3792c2', 'request_id': 'req-030ab098-2dcc-4dfa-b798-90d80c0e817a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:21.227127 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-030ab098-2dcc-4dfa-b798-90d80c0e817a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a60c451fe8914ae3bfcecb70dd3792c2', 'request_id': 'req-030ab098-2dcc-4dfa-b798-90d80c0e817a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:21.238281 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-030ab098-2dcc-4dfa-b798-90d80c0e817a admin admin] The token cache is being invalidate because project a60c451fe8914ae3bfcecb70dd3792c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:43:21.238570 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-030ab098-2dcc-4dfa-b798-90d80c0e817a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:21.239213 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-030ab098-2dcc-4dfa-b798-90d80c0e817a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:21.240754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1040/2069] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:43:20 2022] DELETE /identity/v3/projects/a60c451fe8914ae3bfcecb70dd3792c2 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:21.244941 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c908d-a09b-4833-8512-d2fd2deb7aed service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:21.255109 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c908d-a09b-4833-8512-d2fd2deb7aed service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:21.314186 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a33f7-659e-4e1e-a4d0-02491dfd1709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:21.317525 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a33f7-659e-4e1e-a4d0-02491dfd1709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:21.317525 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a33f7-659e-4e1e-a4d0-02491dfd1709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:21.317525 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a33f7-659e-4e1e-a4d0-02491dfd1709 admin admin] PATH_INFO: `/v3/users/6d4097c02c4240db9f6aa6b4ce1f9a0c` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:21.317525 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a33f7-659e-4e1e-a4d0-02491dfd1709 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d4097c02c4240db9f6aa6b4ce1f9a0c)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:21.318542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a33f7-659e-4e1e-a4d0-02491dfd1709 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:21.502417 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-828a33f7-659e-4e1e-a4d0-02491dfd1709 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d4097c02c4240db9f6aa6b4ce1f9a0c'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:21.503081 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-828a33f7-659e-4e1e-a4d0-02491dfd1709 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d4097c02c4240db9f6aa6b4ce1f9a0c'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:21.503674 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-828a33f7-659e-4e1e-a4d0-02491dfd1709 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d4097c02c4240db9f6aa6b4ce1f9a0c'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:21.510862 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-828a33f7-659e-4e1e-a4d0-02491dfd1709 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d4097c02c4240db9f6aa6b4ce1f9a0c'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:21.590865 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-828a33f7-659e-4e1e-a4d0-02491dfd1709 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d4097c02c4240db9f6aa6b4ce1f9a0c'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:21.614703 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1030/2070] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:43:21 2022] DELETE /identity/v3/users/6d4097c02c4240db9f6aa6b4ce1f9a0c => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:22.089883 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ab098-2dcc-4dfa-b798-90d80c0e817a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:22.091395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-030ab098-2dcc-4dfa-b798-90d80c0e817a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:22.107293 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ab098-2dcc-4dfa-b798-90d80c0e817a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:22.166715 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066127e2-b4d4-4a7d-94a8-636c7f825680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:22.167514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066127e2-b4d4-4a7d-94a8-636c7f825680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:22.167802 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066127e2-b4d4-4a7d-94a8-636c7f825680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:22.168078 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066127e2-b4d4-4a7d-94a8-636c7f825680 admin admin] PATH_INFO: `/v3/projects/7f5970ea97a74badb44f77e733f0efe9` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:22.169276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066127e2-b4d4-4a7d-94a8-636c7f825680 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f5970ea97a74badb44f77e733f0efe9)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:22.171918 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066127e2-b4d4-4a7d-94a8-636c7f825680 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:22.183591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:22.299053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-066127e2-b4d4-4a7d-94a8-636c7f825680 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f5970ea97a74badb44f77e733f0efe9', 'request_id': 'req-066127e2-b4d4-4a7d-94a8-636c7f825680'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:22.389694 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-066127e2-b4d4-4a7d-94a8-636c7f825680 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f5970ea97a74badb44f77e733f0efe9', 'request_id': 'req-066127e2-b4d4-4a7d-94a8-636c7f825680'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:22.396793 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-066127e2-b4d4-4a7d-94a8-636c7f825680 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f5970ea97a74badb44f77e733f0efe9', 'request_id': 'req-066127e2-b4d4-4a7d-94a8-636c7f825680'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:22.407381 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-066127e2-b4d4-4a7d-94a8-636c7f825680 admin admin] The token cache is being invalidate because project 7f5970ea97a74badb44f77e733f0efe9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:43:22.407681 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-066127e2-b4d4-4a7d-94a8-636c7f825680 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:22.412650 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-066127e2-b4d4-4a7d-94a8-636c7f825680 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:22.415092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1041/2071] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:43:22 2022] DELETE /identity/v3/projects/7f5970ea97a74badb44f77e733f0efe9 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:22.421977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15961a03-16df-43a9-96bd-c92876766562 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:22.421977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15961a03-16df-43a9-96bd-c92876766562 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:22.421977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15961a03-16df-43a9-96bd-c92876766562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:22.442768 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-15961a03-16df-43a9-96bd-c92876766562 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:22.445073 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-15961a03-16df-43a9-96bd-c92876766562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:22.474665 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066127e2-b4d4-4a7d-94a8-636c7f825680 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:22.476269 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-066127e2-b4d4-4a7d-94a8-636c7f825680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:22.504414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:22.523626 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1031/2072] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:22.525515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93924e4b-02a6-49f3-bf77-392a454c058e None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:22.525876 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93924e4b-02a6-49f3-bf77-392a454c058e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:22.526144 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93924e4b-02a6-49f3-bf77-392a454c058e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:22.545600 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-93924e4b-02a6-49f3-bf77-392a454c058e None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:22.545880 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066127e2-b4d4-4a7d-94a8-636c7f825680 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:22.546801 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-93924e4b-02a6-49f3-bf77-392a454c058e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:22.550975 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e52ab1-282a-493b-a9f7-50e9ba77e3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:22.551638 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e52ab1-282a-493b-a9f7-50e9ba77e3ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:22.551921 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e52ab1-282a-493b-a9f7-50e9ba77e3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:22.552196 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e52ab1-282a-493b-a9f7-50e9ba77e3ad admin admin] PATH_INFO: `/v3/users/b111c4ab9ff0443b8e0e2334b415a1b7` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:22.553376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e52ab1-282a-493b-a9f7-50e9ba77e3ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=b111c4ab9ff0443b8e0e2334b415a1b7)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:22.555110 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1032/2073] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:43:22.555449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e52ab1-282a-493b-a9f7-50e9ba77e3ad admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:22.559390 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93924e4b-02a6-49f3-bf77-392a454c058e None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:22.569275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93924e4b-02a6-49f3-bf77-392a454c058e None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:22.573929 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509d3b6-5bb3-4b9a-a0c1-5215ca30249e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:22.574518 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509d3b6-5bb3-4b9a-a0c1-5215ca30249e admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:22.574793 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509d3b6-5bb3-4b9a-a0c1-5215ca30249e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:22.575066 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509d3b6-5bb3-4b9a-a0c1-5215ca30249e admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:22.575705 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509d3b6-5bb3-4b9a-a0c1-5215ca30249e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:43:22.576003 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509d3b6-5bb3-4b9a-a0c1-5215ca30249e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:22.577646 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509d3b6-5bb3-4b9a-a0c1-5215ca30249e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:22.590262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1033/2074] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:43:22.594368 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509d3b6-5bb3-4b9a-a0c1-5215ca30249e admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:22.604137 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509d3b6-5bb3-4b9a-a0c1-5215ca30249e admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:22.608812 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8c912-2fff-4f83-aa32-120ee5873ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:22.609425 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8c912-2fff-4f83-aa32-120ee5873ca4 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:22.609757 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8c912-2fff-4f83-aa32-120ee5873ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:22.610026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8c912-2fff-4f83-aa32-120ee5873ca4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:22.610723 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be8c912-2fff-4f83-aa32-120ee5873ca4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:22.612646 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be8c912-2fff-4f83-aa32-120ee5873ca4 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:22.717232 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1034/2075] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:43:22 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:43:22.721294 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8c912-2fff-4f83-aa32-120ee5873ca4 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:22.780915 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8c912-2fff-4f83-aa32-120ee5873ca4 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:22.785709 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086edd58-57db-499a-aeec-cf4bf3a6531e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:22.785845 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-89e52ab1-282a-493b-a9f7-50e9ba77e3ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b111c4ab9ff0443b8e0e2334b415a1b7'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:22.786311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086edd58-57db-499a-aeec-cf4bf3a6531e admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:22.786451 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-89e52ab1-282a-493b-a9f7-50e9ba77e3ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b111c4ab9ff0443b8e0e2334b415a1b7'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:22.786588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086edd58-57db-499a-aeec-cf4bf3a6531e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:22.786857 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086edd58-57db-499a-aeec-cf4bf3a6531e admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:22.787055 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-89e52ab1-282a-493b-a9f7-50e9ba77e3ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b111c4ab9ff0443b8e0e2334b415a1b7'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:22.787470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086edd58-57db-499a-aeec-cf4bf3a6531e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:22.789461 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086edd58-57db-499a-aeec-cf4bf3a6531e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:22.794164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-89e52ab1-282a-493b-a9f7-50e9ba77e3ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b111c4ab9ff0443b8e0e2334b415a1b7'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:22.874974 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-89e52ab1-282a-493b-a9f7-50e9ba77e3ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b111c4ab9ff0443b8e0e2334b415a1b7'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:22.893436 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1042/2076] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:43:22 2022] DELETE /identity/v3/users/b111c4ab9ff0443b8e0e2334b415a1b7 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:43:22.897818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a345c-7c65-43ef-985f-fdc18eaa8208 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:22.898122 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a345c-7c65-43ef-985f-fdc18eaa8208 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:22.898381 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a345c-7c65-43ef-985f-fdc18eaa8208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:22.909826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1035/2077] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:43:22 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:43:22.913719 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086edd58-57db-499a-aeec-cf4bf3a6531e admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:22.915354 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-086edd58-57db-499a-aeec-cf4bf3a6531e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:22.917414 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-677a345c-7c65-43ef-985f-fdc18eaa8208 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:22.918748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-677a345c-7c65-43ef-985f-fdc18eaa8208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:22.927719 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086edd58-57db-499a-aeec-cf4bf3a6531e admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:22.977976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1043/2078] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:43:22.985668 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2cf1b-abbf-433f-8916-073409116fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:22.986291 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2cf1b-abbf-433f-8916-073409116fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:22.986574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2cf1b-abbf-433f-8916-073409116fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:22.986845 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2cf1b-abbf-433f-8916-073409116fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:22.987485 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2cf1b-abbf-433f-8916-073409116fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:22.988840 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a345c-7c65-43ef-985f-fdc18eaa8208 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:22.989425 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2cf1b-abbf-433f-8916-073409116fb0 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:22.990251 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-677a345c-7c65-43ef-985f-fdc18eaa8208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:23.003125 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1036/2079] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:43:23.004371 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2cf1b-abbf-433f-8916-073409116fb0 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:23.013792 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2cf1b-abbf-433f-8916-073409116fb0 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:23.018574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d72736-624a-4a45-9056-b51e67271a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:23.019163 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d72736-624a-4a45-9056-b51e67271a6f admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:23.019440 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d72736-624a-4a45-9056-b51e67271a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:23.019710 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d72736-624a-4a45-9056-b51e67271a6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:23.020366 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d72736-624a-4a45-9056-b51e67271a6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:23.021993 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d72736-624a-4a45-9056-b51e67271a6f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:23.039337 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1037/2080] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:43:23.039337 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d72736-624a-4a45-9056-b51e67271a6f admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:23.047209 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d72736-624a-4a45-9056-b51e67271a6f admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:23.048663 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a345c-7c65-43ef-985f-fdc18eaa8208 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:23.052866 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce017bc-a1fb-4428-8590-cb0f3f647246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:23.053020 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d202fd9-ccae-4896-ad28-b706808e4f99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:23.053592 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce017bc-a1fb-4428-8590-cb0f3f647246 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:23.053708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d202fd9-ccae-4896-ad28-b706808e4f99 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:23.053807 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce017bc-a1fb-4428-8590-cb0f3f647246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:23.053901 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d202fd9-ccae-4896-ad28-b706808e4f99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:23.054065 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce017bc-a1fb-4428-8590-cb0f3f647246 admin admin] PATH_INFO: `/v3/projects/56e8e5711fd94448a4a3eab5832c47da/users/68b73981988b47888f0628ee9ed2aff3/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:23.054181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d202fd9-ccae-4896-ad28-b706808e4f99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:23.061833 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:23.067390 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d202fd9-ccae-4896-ad28-b706808e4f99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:23.069210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d202fd9-ccae-4896-ad28-b706808e4f99 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:23.071124 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce017bc-a1fb-4428-8590-cb0f3f647246 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56e8e5711fd94448a4a3eab5832c47da, user_id=68b73981988b47888f0628ee9ed2aff3, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:23.074053 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce017bc-a1fb-4428-8590-cb0f3f647246 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:23.082268 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1044/2081] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:43:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:43:23.152150 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1038/2082] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:43:23 2022] PUT /identity/v3/projects/56e8e5711fd94448a4a3eab5832c47da/users/68b73981988b47888f0628ee9ed2aff3/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:43:23.165324 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ff066-9e51-4f66-b90b-4244827dd64a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:23.165847 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ff066-9e51-4f66-b90b-4244827dd64a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:23.166039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ff066-9e51-4f66-b90b-4244827dd64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:23.196226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-dc8ff066-9e51-4f66-b90b-4244827dd64a None None] MFA Rules not processed for user `68b73981988b47888f0628ee9ed2aff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:23.197457 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-dc8ff066-9e51-4f66-b90b-4244827dd64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:23.262736 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1045/2083] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:23.274746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce017bc-a1fb-4428-8590-cb0f3f647246 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:23.276663 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-bce017bc-a1fb-4428-8590-cb0f3f647246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:23.340186 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce017bc-a1fb-4428-8590-cb0f3f647246 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:23.343948 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181fe30-0b8f-4400-841f-fc4f9b765c9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:23.344745 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181fe30-0b8f-4400-841f-fc4f9b765c9d service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:23.345062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181fe30-0b8f-4400-841f-fc4f9b765c9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:23.345609 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181fe30-0b8f-4400-841f-fc4f9b765c9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:23.354555 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181fe30-0b8f-4400-841f-fc4f9b765c9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:23.355984 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181fe30-0b8f-4400-841f-fc4f9b765c9d service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:23.365708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1039/2084] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:43:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:43:23.385983 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ff066-9e51-4f66-b90b-4244827dd64a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:23.396415 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:23.456630 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ff066-9e51-4f66-b90b-4244827dd64a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:23.461263 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f872b9-bfbc-4590-946d-cf8990312688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:23.465267 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f872b9-bfbc-4590-946d-cf8990312688 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:23.465267 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f872b9-bfbc-4590-946d-cf8990312688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:23.465267 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f872b9-bfbc-4590-946d-cf8990312688 admin admin] PATH_INFO: `/v3/users/68b73981988b47888f0628ee9ed2aff3` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:23.465267 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f872b9-bfbc-4590-946d-cf8990312688 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68b73981988b47888f0628ee9ed2aff3)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:23.465914 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f872b9-bfbc-4590-946d-cf8990312688 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:23.537498 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181fe30-0b8f-4400-841f-fc4f9b765c9d service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:23.539175 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-9181fe30-0b8f-4400-841f-fc4f9b765c9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:23.555538 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181fe30-0b8f-4400-841f-fc4f9b765c9d service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:23.560564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc5300-381d-476e-b7cb-2b91a6afba0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:23.561499 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc5300-381d-476e-b7cb-2b91a6afba0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:23.562030 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc5300-381d-476e-b7cb-2b91a6afba0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:23.562548 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc5300-381d-476e-b7cb-2b91a6afba0f admin admin] PATH_INFO: `/v3/projects/f568ee82e48c4f729afa5aa56534803e` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:23.564171 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc5300-381d-476e-b7cb-2b91a6afba0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f568ee82e48c4f729afa5aa56534803e)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:23.567068 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc5300-381d-476e-b7cb-2b91a6afba0f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:23.624231 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:23.683704 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-90f872b9-bfbc-4590-946d-cf8990312688 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68b73981988b47888f0628ee9ed2aff3'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:23.684361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-90f872b9-bfbc-4590-946d-cf8990312688 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68b73981988b47888f0628ee9ed2aff3'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:23.684984 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-90f872b9-bfbc-4590-946d-cf8990312688 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68b73981988b47888f0628ee9ed2aff3'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:23.694022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-90f872b9-bfbc-4590-946d-cf8990312688 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68b73981988b47888f0628ee9ed2aff3'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:23.714056 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f4bc5300-381d-476e-b7cb-2b91a6afba0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f568ee82e48c4f729afa5aa56534803e', 'request_id': 'req-f4bc5300-381d-476e-b7cb-2b91a6afba0f'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:23.720499 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f4bc5300-381d-476e-b7cb-2b91a6afba0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f568ee82e48c4f729afa5aa56534803e', 'request_id': 'req-f4bc5300-381d-476e-b7cb-2b91a6afba0f'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:23.726243 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f4bc5300-381d-476e-b7cb-2b91a6afba0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f568ee82e48c4f729afa5aa56534803e', 'request_id': 'req-f4bc5300-381d-476e-b7cb-2b91a6afba0f'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:23.802097 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-90f872b9-bfbc-4590-946d-cf8990312688 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68b73981988b47888f0628ee9ed2aff3'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:23.821064 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1046/2085] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:43:23 2022] DELETE /identity/v3/users/68b73981988b47888f0628ee9ed2aff3 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:23.825189 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68441e88-cfee-42c9-ab03-4a502149c4f3 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:23.825474 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68441e88-cfee-42c9-ab03-4a502149c4f3 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:23.825744 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68441e88-cfee-42c9-ab03-4a502149c4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:23.844268 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-68441e88-cfee-42c9-ab03-4a502149c4f3 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:23.845516 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-68441e88-cfee-42c9-ab03-4a502149c4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:23.878155 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f4bc5300-381d-476e-b7cb-2b91a6afba0f admin admin] The token cache is being invalidate because project f568ee82e48c4f729afa5aa56534803e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:43:23.878453 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f4bc5300-381d-476e-b7cb-2b91a6afba0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:23.879083 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f4bc5300-381d-476e-b7cb-2b91a6afba0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:23.881236 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1040/2086] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:43:23 2022] DELETE /identity/v3/projects/f568ee82e48c4f729afa5aa56534803e => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:23.920721 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1047/2087] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:43:23.932597 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc5300-381d-476e-b7cb-2b91a6afba0f admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:23.934479 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f4bc5300-381d-476e-b7cb-2b91a6afba0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:24.011071 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc5300-381d-476e-b7cb-2b91a6afba0f admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:24.066645 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:24.078389 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1cc3c-5cb2-447b-8bea-b3d2e173d1e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:24.078389 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1cc3c-5cb2-447b-8bea-b3d2e173d1e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:24.078389 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1cc3c-5cb2-447b-8bea-b3d2e173d1e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:24.078389 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1cc3c-5cb2-447b-8bea-b3d2e173d1e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:24.086642 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f1cc3c-5cb2-447b-8bea-b3d2e173d1e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:24.088159 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f1cc3c-5cb2-447b-8bea-b3d2e173d1e4 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:24.098645 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1041/2088] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:43:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:43:24.695011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68441e88-cfee-42c9-ab03-4a502149c4f3 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:24.696469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-68441e88-cfee-42c9-ab03-4a502149c4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:24.709160 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68441e88-cfee-42c9-ab03-4a502149c4f3 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:24.714095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37e46a-9c1f-4a54-b06f-9f457efe0042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:24.715020 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37e46a-9c1f-4a54-b06f-9f457efe0042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:24.715504 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37e46a-9c1f-4a54-b06f-9f457efe0042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:24.715962 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37e46a-9c1f-4a54-b06f-9f457efe0042 admin admin] PATH_INFO: `/v3/projects/56e8e5711fd94448a4a3eab5832c47da` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:24.717333 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d37e46a-9c1f-4a54-b06f-9f457efe0042 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56e8e5711fd94448a4a3eab5832c47da)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:24.720195 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d37e46a-9c1f-4a54-b06f-9f457efe0042 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:24.831785 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6d37e46a-9c1f-4a54-b06f-9f457efe0042 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56e8e5711fd94448a4a3eab5832c47da', 'request_id': 'req-6d37e46a-9c1f-4a54-b06f-9f457efe0042'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:24.932977 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6d37e46a-9c1f-4a54-b06f-9f457efe0042 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56e8e5711fd94448a4a3eab5832c47da', 'request_id': 'req-6d37e46a-9c1f-4a54-b06f-9f457efe0042'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:24.940195 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6d37e46a-9c1f-4a54-b06f-9f457efe0042 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56e8e5711fd94448a4a3eab5832c47da', 'request_id': 'req-6d37e46a-9c1f-4a54-b06f-9f457efe0042'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:24.945094 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:24.951127 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6d37e46a-9c1f-4a54-b06f-9f457efe0042 admin admin] The token cache is being invalidate because project 56e8e5711fd94448a4a3eab5832c47da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:43:24.951418 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6d37e46a-9c1f-4a54-b06f-9f457efe0042 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:24.952073 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6d37e46a-9c1f-4a54-b06f-9f457efe0042 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:24.953671 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1048/2089] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:43:24 2022] DELETE /identity/v3/projects/56e8e5711fd94448a4a3eab5832c47da => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:24.958482 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704103e-61e6-498a-b262-3e8d31197452 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:24.958956 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704103e-61e6-498a-b262-3e8d31197452 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:24.959062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704103e-61e6-498a-b262-3e8d31197452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:24.980528 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-3704103e-61e6-498a-b262-3e8d31197452 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:24.981762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-3704103e-61e6-498a-b262-3e8d31197452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:25.040301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1042/2090] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:43:25.048619 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac357f51-a1bd-45d1-b3b6-b95df9a64fb6 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:25.048993 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac357f51-a1bd-45d1-b3b6-b95df9a64fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:25.049376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac357f51-a1bd-45d1-b3b6-b95df9a64fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:25.073926 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ac357f51-a1bd-45d1-b3b6-b95df9a64fb6 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:25.075158 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ac357f51-a1bd-45d1-b3b6-b95df9a64fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:25.083863 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1049/2091] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:25.089398 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704103e-61e6-498a-b262-3e8d31197452 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:25.100555 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704103e-61e6-498a-b262-3e8d31197452 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:25.105329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1fb218-aa42-441e-ae42-06754ea5df50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:25.108649 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1fb218-aa42-441e-ae42-06754ea5df50 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:25.108649 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1fb218-aa42-441e-ae42-06754ea5df50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:25.108649 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1fb218-aa42-441e-ae42-06754ea5df50 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:25.108649 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1fb218-aa42-441e-ae42-06754ea5df50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:43:25.108649 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1fb218-aa42-441e-ae42-06754ea5df50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:25.109076 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1fb218-aa42-441e-ae42-06754ea5df50 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:25.120889 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1043/2092] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:43:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:43:25.125160 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac357f51-a1bd-45d1-b3b6-b95df9a64fb6 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:25.134869 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac357f51-a1bd-45d1-b3b6-b95df9a64fb6 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:25.139720 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d995a06-ae9c-43b0-aa49-7cdb11013dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:25.140315 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d995a06-ae9c-43b0-aa49-7cdb11013dc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:25.140594 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d995a06-ae9c-43b0-aa49-7cdb11013dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:25.140866 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d995a06-ae9c-43b0-aa49-7cdb11013dc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:25.142158 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d995a06-ae9c-43b0-aa49-7cdb11013dc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:25.144110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d995a06-ae9c-43b0-aa49-7cdb11013dc4 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:25.226928 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1050/2093] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:43:25 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:43:25.231812 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1fb218-aa42-441e-ae42-06754ea5df50 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:25.242319 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1fb218-aa42-441e-ae42-06754ea5df50 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:25.300365 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d36147-0d4d-42ee-b37a-b1fd8eff642e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:25.300968 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d36147-0d4d-42ee-b37a-b1fd8eff642e admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:25.301273 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d36147-0d4d-42ee-b37a-b1fd8eff642e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:25.301537 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d36147-0d4d-42ee-b37a-b1fd8eff642e admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:25.302173 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d36147-0d4d-42ee-b37a-b1fd8eff642e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:25.304095 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d36147-0d4d-42ee-b37a-b1fd8eff642e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:25.386492 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1044/2094] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:43:25 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:43:25.389286 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d995a06-ae9c-43b0-aa49-7cdb11013dc4 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:25.400486 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d995a06-ae9c-43b0-aa49-7cdb11013dc4 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:25.405115 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f6cd3a-79e3-4212-b1a4-1521dceb5bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:25.409184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6cd3a-79e3-4212-b1a4-1521dceb5bdd admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:25.409184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6cd3a-79e3-4212-b1a4-1521dceb5bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:25.409184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6cd3a-79e3-4212-b1a4-1521dceb5bdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:25.409184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f6cd3a-79e3-4212-b1a4-1521dceb5bdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:25.409184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f6cd3a-79e3-4212-b1a4-1521dceb5bdd admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:25.420032 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1051/2095] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:43:25.424017 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d36147-0d4d-42ee-b37a-b1fd8eff642e admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:25.434206 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d36147-0d4d-42ee-b37a-b1fd8eff642e admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:25.439129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02230a4-3617-4144-845c-c313f6457e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:25.439820 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02230a4-3617-4144-845c-c313f6457e9c admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:25.440252 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02230a4-3617-4144-845c-c313f6457e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:25.440534 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02230a4-3617-4144-845c-c313f6457e9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:25.441211 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02230a4-3617-4144-845c-c313f6457e9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:25.442803 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02230a4-3617-4144-845c-c313f6457e9c admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:25.453566 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1045/2096] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:43:25.458310 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f6cd3a-79e3-4212-b1a4-1521dceb5bdd admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:25.468142 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f6cd3a-79e3-4212-b1a4-1521dceb5bdd admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:25.474003 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6272f-78f0-482c-b692-a3aa42b34f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:25.480566 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6272f-78f0-482c-b692-a3aa42b34f61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:25.480566 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6272f-78f0-482c-b692-a3aa42b34f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:25.480566 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6272f-78f0-482c-b692-a3aa42b34f61 admin admin] PATH_INFO: `/v3/projects/a2711dc310d84ae78672c455371b1b31/users/caf60044eacd4b32b624cdf8a6ac159d/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:25.490495 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d6272f-78f0-482c-b692-a3aa42b34f61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2711dc310d84ae78672c455371b1b31, user_id=caf60044eacd4b32b624cdf8a6ac159d, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:25.493320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d6272f-78f0-482c-b692-a3aa42b34f61 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:25.585120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1052/2097] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:43:25 2022] PUT /identity/v3/projects/a2711dc310d84ae78672c455371b1b31/users/caf60044eacd4b32b624cdf8a6ac159d/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:25.595749 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14acb8a-6687-4472-82f4-44a0623bc17c None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:25.596037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14acb8a-6687-4472-82f4-44a0623bc17c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:25.596311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14acb8a-6687-4472-82f4-44a0623bc17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:25.623977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-c14acb8a-6687-4472-82f4-44a0623bc17c None None] MFA Rules not processed for user `caf60044eacd4b32b624cdf8a6ac159d`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:25.625197 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c14acb8a-6687-4472-82f4-44a0623bc17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:25.667580 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:25.689941 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1046/2098] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:25.700952 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6272f-78f0-482c-b692-a3aa42b34f61 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:25.702650 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b4d6272f-78f0-482c-b692-a3aa42b34f61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:25.764834 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6272f-78f0-482c-b692-a3aa42b34f61 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:25.769203 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b668a0-ddc6-42fb-b482-e7c5eac05f96 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '627171bf2a2e4a21ba6f800879f8c665', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:25.769820 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b668a0-ddc6-42fb-b482-e7c5eac05f96 service glance] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:25.770100 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b668a0-ddc6-42fb-b482-e7c5eac05f96 service glance] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:25.770371 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b668a0-ddc6-42fb-b482-e7c5eac05f96 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:25.779014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b668a0-ddc6-42fb-b482-e7c5eac05f96 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:25.780525 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b668a0-ddc6-42fb-b482-e7c5eac05f96 service glance] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:25.790094 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1053/2099] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:43:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:43:34.916290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14acb8a-6687-4472-82f4-44a0623bc17c None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:34.917758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c14acb8a-6687-4472-82f4-44a0623bc17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:34.983273 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14acb8a-6687-4472-82f4-44a0623bc17c None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:34.988298 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde5385-656b-47e8-9eb0-1f4d206fdbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:34.988910 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde5385-656b-47e8-9eb0-1f4d206fdbe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:34.989218 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde5385-656b-47e8-9eb0-1f4d206fdbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:34.989485 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde5385-656b-47e8-9eb0-1f4d206fdbe1 admin admin] PATH_INFO: `/v3/users/b8dcfa4d6e9e413e8c6ec70903170106` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:34.990588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde5385-656b-47e8-9eb0-1f4d206fdbe1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8dcfa4d6e9e413e8c6ec70903170106)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:34.992471 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde5385-656b-47e8-9eb0-1f4d206fdbe1 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:35.164828 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ddde5385-656b-47e8-9eb0-1f4d206fdbe1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8dcfa4d6e9e413e8c6ec70903170106'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:35.165504 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ddde5385-656b-47e8-9eb0-1f4d206fdbe1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8dcfa4d6e9e413e8c6ec70903170106'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:35.166105 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ddde5385-656b-47e8-9eb0-1f4d206fdbe1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8dcfa4d6e9e413e8c6ec70903170106'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:35.172923 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ddde5385-656b-47e8-9eb0-1f4d206fdbe1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8dcfa4d6e9e413e8c6ec70903170106'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:35.240810 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ddde5385-656b-47e8-9eb0-1f4d206fdbe1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8dcfa4d6e9e413e8c6ec70903170106'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:35.258997 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1047/2100] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:43:34 2022] DELETE /identity/v3/users/b8dcfa4d6e9e413e8c6ec70903170106 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:35.523855 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b668a0-ddc6-42fb-b482-e7c5eac05f96 service glance] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:35.535500 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b668a0-ddc6-42fb-b482-e7c5eac05f96 service glance] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:35.598091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e6e79-e40f-44f5-89a7-72f77c483257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:35.598717 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e6e79-e40f-44f5-89a7-72f77c483257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:35.598991 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e6e79-e40f-44f5-89a7-72f77c483257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:35.599260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e6e79-e40f-44f5-89a7-72f77c483257 admin admin] PATH_INFO: `/v3/projects/66e29e74cbf54956b2e4f0bc69fae9b9` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:35.600639 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e6e79-e40f-44f5-89a7-72f77c483257 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66e29e74cbf54956b2e4f0bc69fae9b9)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:35.602595 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e6e79-e40f-44f5-89a7-72f77c483257 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:35.686928 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:35.695066 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-4b4e6e79-e40f-44f5-89a7-72f77c483257 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66e29e74cbf54956b2e4f0bc69fae9b9', 'request_id': 'req-4b4e6e79-e40f-44f5-89a7-72f77c483257'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:35.772253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-4b4e6e79-e40f-44f5-89a7-72f77c483257 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66e29e74cbf54956b2e4f0bc69fae9b9', 'request_id': 'req-4b4e6e79-e40f-44f5-89a7-72f77c483257'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:35.778833 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-4b4e6e79-e40f-44f5-89a7-72f77c483257 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66e29e74cbf54956b2e4f0bc69fae9b9', 'request_id': 'req-4b4e6e79-e40f-44f5-89a7-72f77c483257'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:35.789518 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-4b4e6e79-e40f-44f5-89a7-72f77c483257 admin admin] The token cache is being invalidate because project 66e29e74cbf54956b2e4f0bc69fae9b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:43:35.789821 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-4b4e6e79-e40f-44f5-89a7-72f77c483257 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:35.790484 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-4b4e6e79-e40f-44f5-89a7-72f77c483257 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:35.792183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1054/2101] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:43:35 2022] DELETE /identity/v3/projects/66e29e74cbf54956b2e4f0bc69fae9b9 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:35.796062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde5385-656b-47e8-9eb0-1f4d206fdbe1 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:35.797628 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ddde5385-656b-47e8-9eb0-1f4d206fdbe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:35.811620 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde5385-656b-47e8-9eb0-1f4d206fdbe1 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:35.895695 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7b402b-52de-4dbe-b64b-af11838c3807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:35.896328 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b402b-52de-4dbe-b64b-af11838c3807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:35.896604 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b402b-52de-4dbe-b64b-af11838c3807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:35.896879 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b402b-52de-4dbe-b64b-af11838c3807 admin admin] PATH_INFO: `/v3/users/27595f96542c46afb10a9ccd6c921f33` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:35.898107 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7b402b-52de-4dbe-b64b-af11838c3807 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27595f96542c46afb10a9ccd6c921f33)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:35.900032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7b402b-52de-4dbe-b64b-af11838c3807 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:35.902388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:36.033589 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4d7b402b-52de-4dbe-b64b-af11838c3807 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27595f96542c46afb10a9ccd6c921f33'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:36.034273 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4d7b402b-52de-4dbe-b64b-af11838c3807 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27595f96542c46afb10a9ccd6c921f33'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:36.034919 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4d7b402b-52de-4dbe-b64b-af11838c3807 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27595f96542c46afb10a9ccd6c921f33'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:36.042040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4d7b402b-52de-4dbe-b64b-af11838c3807 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27595f96542c46afb10a9ccd6c921f33'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:36.093406 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4d7b402b-52de-4dbe-b64b-af11838c3807 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27595f96542c46afb10a9ccd6c921f33'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:36.114708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1048/2102] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:43:35 2022] DELETE /identity/v3/users/27595f96542c46afb10a9ccd6c921f33 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:36.340245 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e6e79-e40f-44f5-89a7-72f77c483257 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:36.341713 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4b4e6e79-e40f-44f5-89a7-72f77c483257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:36.357882 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e6e79-e40f-44f5-89a7-72f77c483257 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:36.420502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ca8e1-009c-4b6e-a8bc-03db2e13bf69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:36.421290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ca8e1-009c-4b6e-a8bc-03db2e13bf69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:36.421419 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ca8e1-009c-4b6e-a8bc-03db2e13bf69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:36.421661 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ca8e1-009c-4b6e-a8bc-03db2e13bf69 admin admin] PATH_INFO: `/v3/projects/f55cb27aea574dba90983629a5eb2423` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:36.422783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ca8e1-009c-4b6e-a8bc-03db2e13bf69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f55cb27aea574dba90983629a5eb2423)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:36.425211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ca8e1-009c-4b6e-a8bc-03db2e13bf69 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:36.506028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c79ca8e1-009c-4b6e-a8bc-03db2e13bf69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f55cb27aea574dba90983629a5eb2423', 'request_id': 'req-c79ca8e1-009c-4b6e-a8bc-03db2e13bf69'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:36.562197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c79ca8e1-009c-4b6e-a8bc-03db2e13bf69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f55cb27aea574dba90983629a5eb2423', 'request_id': 'req-c79ca8e1-009c-4b6e-a8bc-03db2e13bf69'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:36.570398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c79ca8e1-009c-4b6e-a8bc-03db2e13bf69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f55cb27aea574dba90983629a5eb2423', 'request_id': 'req-c79ca8e1-009c-4b6e-a8bc-03db2e13bf69'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:36.582443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c79ca8e1-009c-4b6e-a8bc-03db2e13bf69 admin admin] The token cache is being invalidate because project f55cb27aea574dba90983629a5eb2423 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:43:36.582726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c79ca8e1-009c-4b6e-a8bc-03db2e13bf69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:36.583380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c79ca8e1-009c-4b6e-a8bc-03db2e13bf69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:36.584864 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1055/2103] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:43:36 2022] DELETE /identity/v3/projects/f55cb27aea574dba90983629a5eb2423 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:43:36.589785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeaeef4-f6fd-4d0a-bb1c-21693534beb7 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:36.590075 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeaeef4-f6fd-4d0a-bb1c-21693534beb7 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:36.590352 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeaeef4-f6fd-4d0a-bb1c-21693534beb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:36.612655 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-faeaeef4-f6fd-4d0a-bb1c-21693534beb7 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:36.613866 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-faeaeef4-f6fd-4d0a-bb1c-21693534beb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:36.675362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1049/2104] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:36.683085 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2651ba5-81af-464c-acd3-f3f3ecff44df None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:36.683371 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2651ba5-81af-464c-acd3-f3f3ecff44df None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:36.683662 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2651ba5-81af-464c-acd3-f3f3ecff44df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:36.704126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c2651ba5-81af-464c-acd3-f3f3ecff44df None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:36.705461 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c2651ba5-81af-464c-acd3-f3f3ecff44df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:36.713199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1056/2105] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:36.717273 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeaeef4-f6fd-4d0a-bb1c-21693534beb7 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:36.727816 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeaeef4-f6fd-4d0a-bb1c-21693534beb7 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:36.732342 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd608e8-868a-4087-ace8-efc761a969ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:36.732938 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd608e8-868a-4087-ace8-efc761a969ac admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:36.733250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd608e8-868a-4087-ace8-efc761a969ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:36.733527 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd608e8-868a-4087-ace8-efc761a969ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:36.734174 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd608e8-868a-4087-ace8-efc761a969ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:43:36.734477 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd608e8-868a-4087-ace8-efc761a969ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:36.736693 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd608e8-868a-4087-ace8-efc761a969ac admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:36.748413 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1050/2106] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:43:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:43:36.752844 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2651ba5-81af-464c-acd3-f3f3ecff44df None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:36.762459 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2651ba5-81af-464c-acd3-f3f3ecff44df None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:36.768480 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0d275-c566-4085-9808-f8935a8fb5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:36.769330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0d275-c566-4085-9808-f8935a8fb5a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:36.769483 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0d275-c566-4085-9808-f8935a8fb5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:36.769843 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0d275-c566-4085-9808-f8935a8fb5a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:36.770499 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0d275-c566-4085-9808-f8935a8fb5a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:36.772788 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0d275-c566-4085-9808-f8935a8fb5a8 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:36.843443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1057/2107] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:43:36 2022] POST /identity/v3/projects => generated 385 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:43:36.848220 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd608e8-868a-4087-ace8-efc761a969ac admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:36.858869 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd608e8-868a-4087-ace8-efc761a969ac admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:36.918901 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013d9d6-d2ac-465c-9bdc-794fef119bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:36.919512 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013d9d6-d2ac-465c-9bdc-794fef119bb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:36.919791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013d9d6-d2ac-465c-9bdc-794fef119bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:36.920062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013d9d6-d2ac-465c-9bdc-794fef119bb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:36.920678 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013d9d6-d2ac-465c-9bdc-794fef119bb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:36.922613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013d9d6-d2ac-465c-9bdc-794fef119bb8 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:37.008661 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:37.017488 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1051/2108] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:43:36 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:43:37.020142 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0d275-c566-4085-9808-f8935a8fb5a8 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:37.031076 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0d275-c566-4085-9808-f8935a8fb5a8 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:37.036942 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a4ea3-1314-43ed-a113-b050ed18370b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:37.037619 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a4ea3-1314-43ed-a113-b050ed18370b admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:37.037881 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a4ea3-1314-43ed-a113-b050ed18370b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:37.040325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a4ea3-1314-43ed-a113-b050ed18370b admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:37.040325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a4ea3-1314-43ed-a113-b050ed18370b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:37.041004 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a4ea3-1314-43ed-a113-b050ed18370b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:37.053572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1058/2109] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:43:37.057200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013d9d6-d2ac-465c-9bdc-794fef119bb8 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:37.068890 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013d9d6-d2ac-465c-9bdc-794fef119bb8 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:37.074639 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea61d9-1f6c-414b-b0df-535097c93b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:37.074884 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea61d9-1f6c-414b-b0df-535097c93b1b admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:37.075160 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea61d9-1f6c-414b-b0df-535097c93b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:37.075432 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea61d9-1f6c-414b-b0df-535097c93b1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:37.076085 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea61d9-1f6c-414b-b0df-535097c93b1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:37.077943 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea61d9-1f6c-414b-b0df-535097c93b1b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:37.093902 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1052/2110] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:43:37.098308 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a4ea3-1314-43ed-a113-b050ed18370b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:37.109354 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a4ea3-1314-43ed-a113-b050ed18370b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:37.114154 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06139859-f552-4262-8b5f-f642f5af5859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:37.114811 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06139859-f552-4262-8b5f-f642f5af5859 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:37.115089 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06139859-f552-4262-8b5f-f642f5af5859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:37.115359 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06139859-f552-4262-8b5f-f642f5af5859 admin admin] PATH_INFO: `/v3/projects/0c5faa42c27d48879423118aba2005dd/users/5e6fcb79b8814732ba3d4d55b561bc99/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:37.127343 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:37.130380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06139859-f552-4262-8b5f-f642f5af5859 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c5faa42c27d48879423118aba2005dd, user_id=5e6fcb79b8814732ba3d4d55b561bc99, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:37.133166 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06139859-f552-4262-8b5f-f642f5af5859 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:37.219864 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1059/2111] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:43:37 2022] PUT /identity/v3/projects/0c5faa42c27d48879423118aba2005dd/users/5e6fcb79b8814732ba3d4d55b561bc99/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:37.235257 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a46fc88-74d6-42e2-a4e3-d4362c97130e None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:37.235550 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a46fc88-74d6-42e2-a4e3-d4362c97130e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:37.235825 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a46fc88-74d6-42e2-a4e3-d4362c97130e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:37.265616 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-8a46fc88-74d6-42e2-a4e3-d4362c97130e None None] MFA Rules not processed for user `5e6fcb79b8814732ba3d4d55b561bc99`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:37.266901 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8a46fc88-74d6-42e2-a4e3-d4362c97130e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:37.334402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1053/2112] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:37.345222 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06139859-f552-4262-8b5f-f642f5af5859 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:37.347613 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-06139859-f552-4262-8b5f-f642f5af5859 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:37.403563 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06139859-f552-4262-8b5f-f642f5af5859 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:37.407291 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f462b8-2f07-4ce5-a567-7354c3953801 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:37.408561 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f462b8-2f07-4ce5-a567-7354c3953801 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:37.408861 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f462b8-2f07-4ce5-a567-7354c3953801 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:37.409126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f462b8-2f07-4ce5-a567-7354c3953801 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:37.419140 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f462b8-2f07-4ce5-a567-7354c3953801 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:37.420724 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f462b8-2f07-4ce5-a567-7354c3953801 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:37.429723 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1060/2113] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:43:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:43:42.186322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96823820-63be-4d4b-905b-305a7811121e None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:42.186804 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96823820-63be-4d4b-905b-305a7811121e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:42.186896 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96823820-63be-4d4b-905b-305a7811121e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:42.206306 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-96823820-63be-4d4b-905b-305a7811121e None None] MFA Rules not processed for user `32dcdc2e24e44a07ade44d1da330a77e`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:42.208626 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-96823820-63be-4d4b-905b-305a7811121e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:42.260931 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1054/2114] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:46.599027 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f462b8-2f07-4ce5-a567-7354c3953801 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:46.600498 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-02f462b8-2f07-4ce5-a567-7354c3953801 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:46.667519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f462b8-2f07-4ce5-a567-7354c3953801 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:46.672249 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a9117-d183-4cbe-9f4b-629725308582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:46.672875 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a9117-d183-4cbe-9f4b-629725308582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:46.674346 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a9117-d183-4cbe-9f4b-629725308582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:46.674346 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a9117-d183-4cbe-9f4b-629725308582 admin admin] PATH_INFO: `/v3/users/caf60044eacd4b32b624cdf8a6ac159d` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:46.675205 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a9117-d183-4cbe-9f4b-629725308582 admin admin] RBAC: Authorizing `identity:delete_user(user_id=caf60044eacd4b32b624cdf8a6ac159d)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:46.677108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a9117-d183-4cbe-9f4b-629725308582 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:46.821267 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b99a9117-d183-4cbe-9f4b-629725308582 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caf60044eacd4b32b624cdf8a6ac159d'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:46.821838 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b99a9117-d183-4cbe-9f4b-629725308582 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caf60044eacd4b32b624cdf8a6ac159d'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:46.822440 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b99a9117-d183-4cbe-9f4b-629725308582 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caf60044eacd4b32b624cdf8a6ac159d'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:46.829281 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b99a9117-d183-4cbe-9f4b-629725308582 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caf60044eacd4b32b624cdf8a6ac159d'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:46.882105 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b99a9117-d183-4cbe-9f4b-629725308582 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caf60044eacd4b32b624cdf8a6ac159d'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:46.902414 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1061/2115] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:43:46 2022] DELETE /identity/v3/users/caf60044eacd4b32b624cdf8a6ac159d => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:46.909196 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ceef15-eeda-4373-b5f0-d252f22f02f1 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:46.911088 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ceef15-eeda-4373-b5f0-d252f22f02f1 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:46.911088 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ceef15-eeda-4373-b5f0-d252f22f02f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:46.933945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-57ceef15-eeda-4373-b5f0-d252f22f02f1 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:46.933945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-57ceef15-eeda-4373-b5f0-d252f22f02f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:47.003907 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1055/2116] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:47.013793 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a9117-d183-4cbe-9f4b-629725308582 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:47.016037 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b99a9117-d183-4cbe-9f4b-629725308582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:47.030954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:47.080041 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a9117-d183-4cbe-9f4b-629725308582 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:47.083792 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8553a69-9c12-49de-a3b1-0fc74745cdad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:47.084512 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8553a69-9c12-49de-a3b1-0fc74745cdad service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:47.084884 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8553a69-9c12-49de-a3b1-0fc74745cdad service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:47.085289 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8553a69-9c12-49de-a3b1-0fc74745cdad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:47.095742 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8553a69-9c12-49de-a3b1-0fc74745cdad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:47.097478 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8553a69-9c12-49de-a3b1-0fc74745cdad service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:47.110509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1062/2117] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:43:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:43:47.637953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ceef15-eeda-4373-b5f0-d252f22f02f1 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:47.640361 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-57ceef15-eeda-4373-b5f0-d252f22f02f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:47.655438 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ceef15-eeda-4373-b5f0-d252f22f02f1 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:47.661083 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e52ee-3988-4af9-90da-1a88cd6a9123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:47.662363 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e52ee-3988-4af9-90da-1a88cd6a9123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:47.662636 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e52ee-3988-4af9-90da-1a88cd6a9123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:47.662904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e52ee-3988-4af9-90da-1a88cd6a9123 admin admin] PATH_INFO: `/v3/projects/a2711dc310d84ae78672c455371b1b31` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:47.664260 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e52ee-3988-4af9-90da-1a88cd6a9123 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2711dc310d84ae78672c455371b1b31)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:47.666219 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e52ee-3988-4af9-90da-1a88cd6a9123 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:47.684019 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:47.797875 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f52e52ee-3988-4af9-90da-1a88cd6a9123 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2711dc310d84ae78672c455371b1b31', 'request_id': 'req-f52e52ee-3988-4af9-90da-1a88cd6a9123'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:47.804566 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f52e52ee-3988-4af9-90da-1a88cd6a9123 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2711dc310d84ae78672c455371b1b31', 'request_id': 'req-f52e52ee-3988-4af9-90da-1a88cd6a9123'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:47.810186 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f52e52ee-3988-4af9-90da-1a88cd6a9123 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2711dc310d84ae78672c455371b1b31', 'request_id': 'req-f52e52ee-3988-4af9-90da-1a88cd6a9123'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:47.940417 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f52e52ee-3988-4af9-90da-1a88cd6a9123 admin admin] The token cache is being invalidate because project a2711dc310d84ae78672c455371b1b31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:43:47.941565 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f52e52ee-3988-4af9-90da-1a88cd6a9123 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:47.942483 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f52e52ee-3988-4af9-90da-1a88cd6a9123 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:47.944219 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1056/2118] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:43:47 2022] DELETE /identity/v3/projects/a2711dc310d84ae78672c455371b1b31 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:47.949809 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262da3fd-b25a-4261-97b7-17dffeb47be2 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:47.950278 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262da3fd-b25a-4261-97b7-17dffeb47be2 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:47.950535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262da3fd-b25a-4261-97b7-17dffeb47be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:47.972340 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-262da3fd-b25a-4261-97b7-17dffeb47be2 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:47.973589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-262da3fd-b25a-4261-97b7-17dffeb47be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:48.036761 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1063/2119] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:48.045583 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd428e5-28a5-4a53-a6e4-c1eec5895f62 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:48.045878 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd428e5-28a5-4a53-a6e4-c1eec5895f62 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:48.046171 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd428e5-28a5-4a53-a6e4-c1eec5895f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:48.066698 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-1dd428e5-28a5-4a53-a6e4-c1eec5895f62 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:48.067913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1dd428e5-28a5-4a53-a6e4-c1eec5895f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:48.076396 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1057/2120] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:48.081564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262da3fd-b25a-4261-97b7-17dffeb47be2 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:48.091250 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262da3fd-b25a-4261-97b7-17dffeb47be2 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:48.096153 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388ff8c-92d0-4241-acbf-dd55fc90828a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:48.096746 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388ff8c-92d0-4241-acbf-dd55fc90828a admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:48.097019 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388ff8c-92d0-4241-acbf-dd55fc90828a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:48.097393 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388ff8c-92d0-4241-acbf-dd55fc90828a admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:48.097996 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388ff8c-92d0-4241-acbf-dd55fc90828a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:43:48.098293 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388ff8c-92d0-4241-acbf-dd55fc90828a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:48.099867 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388ff8c-92d0-4241-acbf-dd55fc90828a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:48.114189 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1064/2121] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:43:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:43:48.118336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd428e5-28a5-4a53-a6e4-c1eec5895f62 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:48.136005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd428e5-28a5-4a53-a6e4-c1eec5895f62 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:48.138562 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0bbd2-994c-491a-906b-ef1312a928b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:48.139368 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0bbd2-994c-491a-906b-ef1312a928b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:48.139652 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0bbd2-994c-491a-906b-ef1312a928b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:48.140058 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0bbd2-994c-491a-906b-ef1312a928b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:48.140687 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da0bbd2-994c-491a-906b-ef1312a928b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:48.143592 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da0bbd2-994c-491a-906b-ef1312a928b0 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:48.261700 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1058/2122] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:43:48 2022] POST /identity/v3/projects => generated 375 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:43:48.265786 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388ff8c-92d0-4241-acbf-dd55fc90828a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:48.280584 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388ff8c-92d0-4241-acbf-dd55fc90828a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:48.383295 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5005a9a-3631-459b-b5e3-e7154b4c1025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:48.383980 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5005a9a-3631-459b-b5e3-e7154b4c1025 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:48.384265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5005a9a-3631-459b-b5e3-e7154b4c1025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:48.384539 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5005a9a-3631-459b-b5e3-e7154b4c1025 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:48.385225 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5005a9a-3631-459b-b5e3-e7154b4c1025 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:48.387235 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5005a9a-3631-459b-b5e3-e7154b4c1025 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:48.427649 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0bbd2-994c-491a-906b-ef1312a928b0 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:48.429453 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-7da0bbd2-994c-491a-906b-ef1312a928b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:48.443242 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0bbd2-994c-491a-906b-ef1312a928b0 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:48.448663 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff61bc-0cae-4388-9c15-fef4a93aeb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:48.449484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff61bc-0cae-4388-9c15-fef4a93aeb81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:48.449773 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff61bc-0cae-4388-9c15-fef4a93aeb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:48.450047 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff61bc-0cae-4388-9c15-fef4a93aeb81 admin admin] PATH_INFO: `/v3/users/5e6fcb79b8814732ba3d4d55b561bc99` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:48.451291 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff61bc-0cae-4388-9c15-fef4a93aeb81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e6fcb79b8814732ba3d4d55b561bc99)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:48.455557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff61bc-0cae-4388-9c15-fef4a93aeb81 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:48.536379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1065/2123] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:43:48 2022] POST /identity/v3/users => generated 288 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:43:48.540314 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5005a9a-3631-459b-b5e3-e7154b4c1025 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:48.552140 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5005a9a-3631-459b-b5e3-e7154b4c1025 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:48.559984 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a48678-f799-47bb-8ae9-d6a543c7cdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:48.560836 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a48678-f799-47bb-8ae9-d6a543c7cdc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:48.561212 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a48678-f799-47bb-8ae9-d6a543c7cdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:48.561485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a48678-f799-47bb-8ae9-d6a543c7cdc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:48.562160 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a48678-f799-47bb-8ae9-d6a543c7cdc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:48.563727 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a48678-f799-47bb-8ae9-d6a543c7cdc1 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:48.575836 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1066/2124] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:43:48.579874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a48678-f799-47bb-8ae9-d6a543c7cdc1 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:48.589778 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a48678-f799-47bb-8ae9-d6a543c7cdc1 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:48.594849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef894772-d8bb-45b1-84d0-4648ccd91f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:48.595642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef894772-d8bb-45b1-84d0-4648ccd91f7a admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:48.595929 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef894772-d8bb-45b1-84d0-4648ccd91f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:48.596201 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef894772-d8bb-45b1-84d0-4648ccd91f7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:48.596855 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef894772-d8bb-45b1-84d0-4648ccd91f7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:48.598496 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef894772-d8bb-45b1-84d0-4648ccd91f7a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:48.608724 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1067/2125] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:43:48.613497 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef894772-d8bb-45b1-84d0-4648ccd91f7a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:48.623303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef894772-d8bb-45b1-84d0-4648ccd91f7a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:48.629170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964be6d-fb48-45dd-881a-985958fbf8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:48.633116 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964be6d-fb48-45dd-881a-985958fbf8ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:48.633116 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964be6d-fb48-45dd-881a-985958fbf8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:48.633116 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964be6d-fb48-45dd-881a-985958fbf8ba admin admin] PATH_INFO: `/v3/projects/4c1d5017dd944aaf9e2d7b46e0e51185/users/607fe4be34e54961a329eac3afdcced6/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:48.646030 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:48.697588 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4964be6d-fb48-45dd-881a-985958fbf8ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c1d5017dd944aaf9e2d7b46e0e51185, user_id=607fe4be34e54961a329eac3afdcced6, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:48.697694 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d8ff61bc-0cae-4388-9c15-fef4a93aeb81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e6fcb79b8814732ba3d4d55b561bc99'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:48.697694 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d8ff61bc-0cae-4388-9c15-fef4a93aeb81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e6fcb79b8814732ba3d4d55b561bc99'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:48.697694 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d8ff61bc-0cae-4388-9c15-fef4a93aeb81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e6fcb79b8814732ba3d4d55b561bc99'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:48.697897 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4964be6d-fb48-45dd-881a-985958fbf8ba admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:48.726373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d8ff61bc-0cae-4388-9c15-fef4a93aeb81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e6fcb79b8814732ba3d4d55b561bc99'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:48.796389 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d8ff61bc-0cae-4388-9c15-fef4a93aeb81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e6fcb79b8814732ba3d4d55b561bc99'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:48.805182 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1068/2126] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:43:48 2022] PUT /identity/v3/projects/4c1d5017dd944aaf9e2d7b46e0e51185/users/607fe4be34e54961a329eac3afdcced6/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:43:48.813581 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb6f31-6e48-49b5-8b3c-7dc23e6d50f0 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:48.813881 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb6f31-6e48-49b5-8b3c-7dc23e6d50f0 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:48.814818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb6f31-6e48-49b5-8b3c-7dc23e6d50f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:48.821936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1059/2127] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:43:48 2022] DELETE /identity/v3/users/5e6fcb79b8814732ba3d4d55b561bc99 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:43:48.825967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939091f-cd63-4267-bd6a-60d187d28437 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:48.826726 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939091f-cd63-4267-bd6a-60d187d28437 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:48.826726 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939091f-cd63-4267-bd6a-60d187d28437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:48.847916 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-d939091f-cd63-4267-bd6a-60d187d28437 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:48.848554 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8beb6f31-6e48-49b5-8b3c-7dc23e6d50f0 None None] MFA Rules not processed for user `607fe4be34e54961a329eac3afdcced6`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:48.849231 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d939091f-cd63-4267-bd6a-60d187d28437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:48.850022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8beb6f31-6e48-49b5-8b3c-7dc23e6d50f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:48.931589 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:48.940787 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1069/2128] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:43:48.950032 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beb6f31-6e48-49b5-8b3c-7dc23e6d50f0 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:48.951693 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8beb6f31-6e48-49b5-8b3c-7dc23e6d50f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:48.960975 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1060/2129] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:43:48.977560 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939091f-cd63-4267-bd6a-60d187d28437 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:48.980784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d939091f-cd63-4267-bd6a-60d187d28437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:49.044712 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beb6f31-6e48-49b5-8b3c-7dc23e6d50f0 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:49.049287 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e64e8c-4eab-4656-9e2a-fe4c799436d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:49.049988 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e64e8c-4eab-4656-9e2a-fe4c799436d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:49.050263 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e64e8c-4eab-4656-9e2a-fe4c799436d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:49.050528 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e64e8c-4eab-4656-9e2a-fe4c799436d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:49.051636 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939091f-cd63-4267-bd6a-60d187d28437 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:49.055674 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d454bd6-0f22-489d-8c33-9f9928079545 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:49.056353 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d454bd6-0f22-489d-8c33-9f9928079545 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:49.056703 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d454bd6-0f22-489d-8c33-9f9928079545 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:49.057048 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d454bd6-0f22-489d-8c33-9f9928079545 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:49.060231 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e64e8c-4eab-4656-9e2a-fe4c799436d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:49.061750 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e64e8c-4eab-4656-9e2a-fe4c799436d5 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:49.069202 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d454bd6-0f22-489d-8c33-9f9928079545 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:49.070666 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d454bd6-0f22-489d-8c33-9f9928079545 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:49.081495 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1061/2130] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:43:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:43:49.084131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1070/2131] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:43:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:43:49.102795 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945b44a-fd10-4a55-b649-ccef8044ecd0 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:49.103086 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945b44a-fd10-4a55-b649-ccef8044ecd0 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:49.103421 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945b44a-fd10-4a55-b649-ccef8044ecd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:49.129422 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-0945b44a-fd10-4a55-b649-ccef8044ecd0 None None] MFA Rules not processed for user `7f3422c6f9a041ceba851b71bd15d0df`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:49.130734 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0945b44a-fd10-4a55-b649-ccef8044ecd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:49.197709 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1062/2132] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 08:43:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:43:49.247864 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e64e8c-4eab-4656-9e2a-fe4c799436d5 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:49.252494 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b1e64e8c-4eab-4656-9e2a-fe4c799436d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:49.338971 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e64e8c-4eab-4656-9e2a-fe4c799436d5 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:49.346164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40627c3-4d36-40dc-a283-1f6f279d0939 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e69c0e02ad944f2a85fc5318018e5cb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:49.347282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40627c3-4d36-40dc-a283-1f6f279d0939 service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:49.347848 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40627c3-4d36-40dc-a283-1f6f279d0939 service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:49.348348 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40627c3-4d36-40dc-a283-1f6f279d0939 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:49.359103 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40627c3-4d36-40dc-a283-1f6f279d0939 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:49.361045 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40627c3-4d36-40dc-a283-1f6f279d0939 service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:49.370458 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1071/2133] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 08:43:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 08:43:49.539885 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0945b44a-fd10-4a55-b649-ccef8044ecd0 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:49.541515 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0945b44a-fd10-4a55-b649-ccef8044ecd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:49.576613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0945b44a-fd10-4a55-b649-ccef8044ecd0 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:49.596440 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a808274-431e-43ac-958e-c9b5ef5aea7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:49.604303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a808274-431e-43ac-958e-c9b5ef5aea7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:49.604303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a808274-431e-43ac-958e-c9b5ef5aea7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:49.604303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a808274-431e-43ac-958e-c9b5ef5aea7f admin admin] PATH_INFO: `/v3/projects/0c5faa42c27d48879423118aba2005dd` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:49.611827 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a808274-431e-43ac-958e-c9b5ef5aea7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c5faa42c27d48879423118aba2005dd)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:49.615880 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a808274-431e-43ac-958e-c9b5ef5aea7f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:49.787716 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-9a808274-431e-43ac-958e-c9b5ef5aea7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c5faa42c27d48879423118aba2005dd', 'request_id': 'req-9a808274-431e-43ac-958e-c9b5ef5aea7f'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:49.798094 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-9a808274-431e-43ac-958e-c9b5ef5aea7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c5faa42c27d48879423118aba2005dd', 'request_id': 'req-9a808274-431e-43ac-958e-c9b5ef5aea7f'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:49.807832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-9a808274-431e-43ac-958e-c9b5ef5aea7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c5faa42c27d48879423118aba2005dd', 'request_id': 'req-9a808274-431e-43ac-958e-c9b5ef5aea7f'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:49.922233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-9a808274-431e-43ac-958e-c9b5ef5aea7f admin admin] The token cache is being invalidate because project 0c5faa42c27d48879423118aba2005dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:43:49.922233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-9a808274-431e-43ac-958e-c9b5ef5aea7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:49.922233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-9a808274-431e-43ac-958e-c9b5ef5aea7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:49.922233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1063/2134] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:43:49 2022] DELETE /identity/v3/projects/0c5faa42c27d48879423118aba2005dd => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:49.926394 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eec6fe-107d-44a0-8800-a4f72fd59dfa None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:49.926679 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eec6fe-107d-44a0-8800-a4f72fd59dfa None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:49.926948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eec6fe-107d-44a0-8800-a4f72fd59dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:49.963967 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-70eec6fe-107d-44a0-8800-a4f72fd59dfa None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:49.969477 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-70eec6fe-107d-44a0-8800-a4f72fd59dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:49.981406 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:50.111530 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1072/2135] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:50.119388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3186b-a5fb-4490-9a00-85db842fd3cb None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:50.119967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3186b-a5fb-4490-9a00-85db842fd3cb None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:50.120335 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3186b-a5fb-4490-9a00-85db842fd3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:50.149280 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-f9c3186b-a5fb-4490-9a00-85db842fd3cb None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:50.151681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f9c3186b-a5fb-4490-9a00-85db842fd3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:50.164203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1064/2136] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:50.168564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eec6fe-107d-44a0-8800-a4f72fd59dfa None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:50.201175 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eec6fe-107d-44a0-8800-a4f72fd59dfa None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:50.207957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbfa7b-d245-4009-ac91-41a84bf69f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:50.208674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbfa7b-d245-4009-ac91-41a84bf69f33 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:50.209062 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbfa7b-d245-4009-ac91-41a84bf69f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:50.209442 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbfa7b-d245-4009-ac91-41a84bf69f33 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:50.211253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbfa7b-d245-4009-ac91-41a84bf69f33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:43:50.211253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbfa7b-d245-4009-ac91-41a84bf69f33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:50.212796 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbfa7b-d245-4009-ac91-41a84bf69f33 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:50.237932 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1073/2137] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:43:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:43:50.243579 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3186b-a5fb-4490-9a00-85db842fd3cb None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:50.257034 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3186b-a5fb-4490-9a00-85db842fd3cb None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:50.267191 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bb24b-2ca2-43df-a14b-9b4f5d4a6f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:50.268030 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bb24b-2ca2-43df-a14b-9b4f5d4a6f04 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:50.268304 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bb24b-2ca2-43df-a14b-9b4f5d4a6f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:50.272896 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bb24b-2ca2-43df-a14b-9b4f5d4a6f04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:50.273587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8bb24b-2ca2-43df-a14b-9b4f5d4a6f04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:50.275512 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8bb24b-2ca2-43df-a14b-9b4f5d4a6f04 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:50.313821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:50.421872 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1065/2138] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:43:50 2022] POST /identity/v3/projects => generated 361 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:43:50.427451 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbfa7b-d245-4009-ac91-41a84bf69f33 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:50.442678 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbfa7b-d245-4009-ac91-41a84bf69f33 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:50.539335 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a31bc-5029-47e0-87a1-c6ff92974caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:50.539335 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a31bc-5029-47e0-87a1-c6ff92974caa admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:50.539335 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a31bc-5029-47e0-87a1-c6ff92974caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:50.539335 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a31bc-5029-47e0-87a1-c6ff92974caa admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:50.539754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a31bc-5029-47e0-87a1-c6ff92974caa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:50.547037 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a31bc-5029-47e0-87a1-c6ff92974caa admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:50.668444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1074/2139] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:43:50 2022] POST /identity/v3/users => generated 281 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:43:50.672514 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bb24b-2ca2-43df-a14b-9b4f5d4a6f04 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:50.699214 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bb24b-2ca2-43df-a14b-9b4f5d4a6f04 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:50.710694 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214bc008-28da-460f-a321-71ee3a8b6f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:50.711317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214bc008-28da-460f-a321-71ee3a8b6f70 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:50.711591 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214bc008-28da-460f-a321-71ee3a8b6f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:50.720929 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214bc008-28da-460f-a321-71ee3a8b6f70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:50.720929 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214bc008-28da-460f-a321-71ee3a8b6f70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:50.720929 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214bc008-28da-460f-a321-71ee3a8b6f70 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:50.739535 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1066/2140] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:43:50.743414 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a31bc-5029-47e0-87a1-c6ff92974caa admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:50.756861 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a31bc-5029-47e0-87a1-c6ff92974caa admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:50.763032 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c8c0a-f9e8-4251-bd42-59d80d590a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:50.764339 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c8c0a-f9e8-4251-bd42-59d80d590a09 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:50.764760 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c8c0a-f9e8-4251-bd42-59d80d590a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:50.765133 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c8c0a-f9e8-4251-bd42-59d80d590a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:50.766473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6c8c0a-f9e8-4251-bd42-59d80d590a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:50.769085 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6c8c0a-f9e8-4251-bd42-59d80d590a09 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:50.789738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1075/2141] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:43:50.794203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214bc008-28da-460f-a321-71ee3a8b6f70 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:50.832341 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214bc008-28da-460f-a321-71ee3a8b6f70 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:50.867084 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1cc16-5315-4973-b0f0-1ebd7df55cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:50.867084 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1cc16-5315-4973-b0f0-1ebd7df55cc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:50.867084 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1cc16-5315-4973-b0f0-1ebd7df55cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:50.867084 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1cc16-5315-4973-b0f0-1ebd7df55cc7 admin admin] PATH_INFO: `/v3/projects/485a973be18b4423a9dc2acf8831cc1b/users/869c601dbf244402b3a63e406c7764e3/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:50.973569 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec1cc16-5315-4973-b0f0-1ebd7df55cc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=485a973be18b4423a9dc2acf8831cc1b, user_id=869c601dbf244402b3a63e406c7764e3, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:50.973569 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec1cc16-5315-4973-b0f0-1ebd7df55cc7 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:51.050753 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1067/2142] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:43:50 2022] PUT /identity/v3/projects/485a973be18b4423a9dc2acf8831cc1b/users/869c601dbf244402b3a63e406c7764e3/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:43:51.062668 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29565576-dfd7-4d95-8b97-98f0bd5e340b None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:51.063287 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29565576-dfd7-4d95-8b97-98f0bd5e340b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:51.063287 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29565576-dfd7-4d95-8b97-98f0bd5e340b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:51.121618 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-29565576-dfd7-4d95-8b97-98f0bd5e340b None None] MFA Rules not processed for user `869c601dbf244402b3a63e406c7764e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:51.122854 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-29565576-dfd7-4d95-8b97-98f0bd5e340b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:51.272630 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1076/2143] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:51.327704 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1cc16-5315-4973-b0f0-1ebd7df55cc7 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:51.329543 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6ec1cc16-5315-4973-b0f0-1ebd7df55cc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:51.482219 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1cc16-5315-4973-b0f0-1ebd7df55cc7 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:51.497568 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbbdc6-36bd-4c03-975d-569a163fe87f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:51.498302 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbbdc6-36bd-4c03-975d-569a163fe87f service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:51.498662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbbdc6-36bd-4c03-975d-569a163fe87f service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:51.499269 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbbdc6-36bd-4c03-975d-569a163fe87f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:51.515082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbbdc6-36bd-4c03-975d-569a163fe87f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:51.517056 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbbdc6-36bd-4c03-975d-569a163fe87f service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:51.532815 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1068/2144] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:43:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:43:51.549579 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29565576-dfd7-4d95-8b97-98f0bd5e340b None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:51.558043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:51.681285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29565576-dfd7-4d95-8b97-98f0bd5e340b None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:51.694130 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a2c08-1910-4d0b-9224-e3e532c42067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:51.695071 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a2c08-1910-4d0b-9224-e3e532c42067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:51.695480 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a2c08-1910-4d0b-9224-e3e532c42067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:51.695837 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a2c08-1910-4d0b-9224-e3e532c42067 admin admin] PATH_INFO: `/v3/users/869c601dbf244402b3a63e406c7764e3` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:51.698218 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775a2c08-1910-4d0b-9224-e3e532c42067 admin admin] RBAC: Authorizing `identity:delete_user(user_id=869c601dbf244402b3a63e406c7764e3)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:51.700295 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775a2c08-1910-4d0b-9224-e3e532c42067 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:51.888634 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-775a2c08-1910-4d0b-9224-e3e532c42067 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '869c601dbf244402b3a63e406c7764e3'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:51.889702 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-775a2c08-1910-4d0b-9224-e3e532c42067 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '869c601dbf244402b3a63e406c7764e3'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:51.908700 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-775a2c08-1910-4d0b-9224-e3e532c42067 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '869c601dbf244402b3a63e406c7764e3'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:51.920483 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-775a2c08-1910-4d0b-9224-e3e532c42067 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '869c601dbf244402b3a63e406c7764e3'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:51.973679 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-775a2c08-1910-4d0b-9224-e3e532c42067 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '869c601dbf244402b3a63e406c7764e3'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:52.012131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1077/2145] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:43:51 2022] DELETE /identity/v3/users/869c601dbf244402b3a63e406c7764e3 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:52.015598 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f483d-e43e-4fad-b5c5-11799f1116c1 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:52.015890 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f483d-e43e-4fad-b5c5-11799f1116c1 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:52.016307 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f483d-e43e-4fad-b5c5-11799f1116c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:52.041950 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-2e2f483d-e43e-4fad-b5c5-11799f1116c1 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:52.044691 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2e2f483d-e43e-4fad-b5c5-11799f1116c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:52.127744 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1069/2146] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:52.137838 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a2c08-1910-4d0b-9224-e3e532c42067 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:52.139364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-775a2c08-1910-4d0b-9224-e3e532c42067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:52.229293 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a2c08-1910-4d0b-9224-e3e532c42067 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:52.234589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f15db2f-e433-427a-a787-f86809fbacab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:52.235516 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15db2f-e433-427a-a787-f86809fbacab service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:52.236050 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15db2f-e433-427a-a787-f86809fbacab service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:52.237284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15db2f-e433-427a-a787-f86809fbacab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:52.279848 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f15db2f-e433-427a-a787-f86809fbacab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:52.298710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f15db2f-e433-427a-a787-f86809fbacab service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:52.315140 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1078/2147] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:43:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:43:52.960179 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f483d-e43e-4fad-b5c5-11799f1116c1 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:52.960179 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2e2f483d-e43e-4fad-b5c5-11799f1116c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:53.003844 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f483d-e43e-4fad-b5c5-11799f1116c1 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:53.016053 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b8101-e89a-45c6-8490-9716df574698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:53.017877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b8101-e89a-45c6-8490-9716df574698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:53.019275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b8101-e89a-45c6-8490-9716df574698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:53.020917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b8101-e89a-45c6-8490-9716df574698 admin admin] PATH_INFO: `/v3/projects/485a973be18b4423a9dc2acf8831cc1b` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:53.022332 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b8101-e89a-45c6-8490-9716df574698 admin admin] RBAC: Authorizing `identity:delete_project(project_id=485a973be18b4423a9dc2acf8831cc1b)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:53.024538 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b8101-e89a-45c6-8490-9716df574698 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:53.156323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-753b8101-e89a-45c6-8490-9716df574698 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '485a973be18b4423a9dc2acf8831cc1b', 'request_id': 'req-753b8101-e89a-45c6-8490-9716df574698'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:53.166164 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-753b8101-e89a-45c6-8490-9716df574698 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '485a973be18b4423a9dc2acf8831cc1b', 'request_id': 'req-753b8101-e89a-45c6-8490-9716df574698'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:53.167586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:53.172498 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-753b8101-e89a-45c6-8490-9716df574698 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '485a973be18b4423a9dc2acf8831cc1b', 'request_id': 'req-753b8101-e89a-45c6-8490-9716df574698'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:53.240189 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-753b8101-e89a-45c6-8490-9716df574698 admin admin] The token cache is being invalidate because project 485a973be18b4423a9dc2acf8831cc1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:43:53.240572 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-753b8101-e89a-45c6-8490-9716df574698 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:53.241261 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-753b8101-e89a-45c6-8490-9716df574698 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:53.242880 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1070/2148] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:43:52 2022] DELETE /identity/v3/projects/485a973be18b4423a9dc2acf8831cc1b => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:53.251049 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e71efd-0dec-4520-a297-d5ba727aa587 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:53.251699 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e71efd-0dec-4520-a297-d5ba727aa587 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:53.251794 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e71efd-0dec-4520-a297-d5ba727aa587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:53.275993 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e1e71efd-0dec-4520-a297-d5ba727aa587 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:53.282091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e1e71efd-0dec-4520-a297-d5ba727aa587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:53.324554 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:53.373271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1079/2149] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:53.382594 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a5f65-dda1-4b46-9c3d-306c76841fd6 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:53.382912 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a5f65-dda1-4b46-9c3d-306c76841fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:53.383185 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a5f65-dda1-4b46-9c3d-306c76841fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:53.402574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-8d3a5f65-dda1-4b46-9c3d-306c76841fd6 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:53.403794 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8d3a5f65-dda1-4b46-9c3d-306c76841fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:53.424612 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1071/2150] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:53.429026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e71efd-0dec-4520-a297-d5ba727aa587 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:53.441781 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e71efd-0dec-4520-a297-d5ba727aa587 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:53.446495 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd54382-ebc7-498c-bfd3-3a4d95742ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:53.447120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd54382-ebc7-498c-bfd3-3a4d95742ca0 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:53.447403 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd54382-ebc7-498c-bfd3-3a4d95742ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:53.447674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd54382-ebc7-498c-bfd3-3a4d95742ca0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:53.448332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd54382-ebc7-498c-bfd3-3a4d95742ca0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:43:53.448638 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd54382-ebc7-498c-bfd3-3a4d95742ca0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:53.450282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd54382-ebc7-498c-bfd3-3a4d95742ca0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:53.463354 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1080/2151] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:43:53.467891 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a5f65-dda1-4b46-9c3d-306c76841fd6 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:53.479043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a5f65-dda1-4b46-9c3d-306c76841fd6 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:53.484237 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a2220-a770-4276-b3a4-ad3af0d57683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:53.484896 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a2220-a770-4276-b3a4-ad3af0d57683 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:53.485198 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a2220-a770-4276-b3a4-ad3af0d57683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:53.485472 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a2220-a770-4276-b3a4-ad3af0d57683 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:53.486626 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a2220-a770-4276-b3a4-ad3af0d57683 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:53.488689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a2220-a770-4276-b3a4-ad3af0d57683 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:53.833620 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1072/2152] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:43:53 2022] POST /identity/v3/projects => generated 371 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:43:53.835072 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd54382-ebc7-498c-bfd3-3a4d95742ca0 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:53.845339 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd54382-ebc7-498c-bfd3-3a4d95742ca0 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:53.927488 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e272e9-05d7-4c88-8978-a58406e03403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:53.928167 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e272e9-05d7-4c88-8978-a58406e03403 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:53.928449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e272e9-05d7-4c88-8978-a58406e03403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:53.928856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e272e9-05d7-4c88-8978-a58406e03403 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:53.929541 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e272e9-05d7-4c88-8978-a58406e03403 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:53.931735 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e272e9-05d7-4c88-8978-a58406e03403 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:53.963161 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a2220-a770-4276-b3a4-ad3af0d57683 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:53.965065 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1a8a2220-a770-4276-b3a4-ad3af0d57683 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:53.988520 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a2220-a770-4276-b3a4-ad3af0d57683 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:53.998935 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241246ca-cdf0-4aaa-a38d-35964c0ca265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:53.999813 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241246ca-cdf0-4aaa-a38d-35964c0ca265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:54.000912 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241246ca-cdf0-4aaa-a38d-35964c0ca265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:54.001398 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241246ca-cdf0-4aaa-a38d-35964c0ca265 admin admin] PATH_INFO: `/v3/users/607fe4be34e54961a329eac3afdcced6` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:54.004613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241246ca-cdf0-4aaa-a38d-35964c0ca265 admin admin] RBAC: Authorizing `identity:delete_user(user_id=607fe4be34e54961a329eac3afdcced6)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:54.008264 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241246ca-cdf0-4aaa-a38d-35964c0ca265 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:54.038192 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1081/2153] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:43:53 2022] POST /identity/v3/users => generated 286 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:43:54.042627 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e272e9-05d7-4c88-8978-a58406e03403 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:54.061541 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e272e9-05d7-4c88-8978-a58406e03403 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:54.067791 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809740cc-2bca-4a27-962b-263e044c972e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:54.068523 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809740cc-2bca-4a27-962b-263e044c972e admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:54.069077 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809740cc-2bca-4a27-962b-263e044c972e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:54.073005 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809740cc-2bca-4a27-962b-263e044c972e admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:54.074367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809740cc-2bca-4a27-962b-263e044c972e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:54.079775 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809740cc-2bca-4a27-962b-263e044c972e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:54.097773 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1082/2154] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:43:54.105199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809740cc-2bca-4a27-962b-263e044c972e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:54.126518 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809740cc-2bca-4a27-962b-263e044c972e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:54.133538 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c826b5-f7fd-4f0c-bd7b-52304e91171e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:54.134456 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c826b5-f7fd-4f0c-bd7b-52304e91171e admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:54.134943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c826b5-f7fd-4f0c-bd7b-52304e91171e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:54.135428 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c826b5-f7fd-4f0c-bd7b-52304e91171e admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:54.136227 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c826b5-f7fd-4f0c-bd7b-52304e91171e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:54.138211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c826b5-f7fd-4f0c-bd7b-52304e91171e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:54.150658 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1083/2155] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:43:54.154710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c826b5-f7fd-4f0c-bd7b-52304e91171e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:54.183508 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c826b5-f7fd-4f0c-bd7b-52304e91171e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:54.199992 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac3c8fe-fa56-4a32-965a-d2418309dc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:54.201429 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3c8fe-fa56-4a32-965a-d2418309dc23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:54.201914 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3c8fe-fa56-4a32-965a-d2418309dc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:54.202113 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3c8fe-fa56-4a32-965a-d2418309dc23 admin admin] PATH_INFO: `/v3/projects/2f9b093fc72d42798e6e3c3ea42516f2/users/0d9c588e4fa34493b8490215005186ee/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:54.221273 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac3c8fe-fa56-4a32-965a-d2418309dc23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f9b093fc72d42798e6e3c3ea42516f2, user_id=0d9c588e4fa34493b8490215005186ee, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:54.224439 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac3c8fe-fa56-4a32-965a-d2418309dc23 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:54.337179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1084/2156] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:43:54 2022] PUT /identity/v3/projects/2f9b093fc72d42798e6e3c3ea42516f2/users/0d9c588e4fa34493b8490215005186ee/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:54.344471 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02312dff-bc23-41db-aac6-2c3c94abaecc None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:54.346097 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02312dff-bc23-41db-aac6-2c3c94abaecc None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:54.346654 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02312dff-bc23-41db-aac6-2c3c94abaecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:54.390147 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-241246ca-cdf0-4aaa-a38d-35964c0ca265 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '607fe4be34e54961a329eac3afdcced6'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:54.395114 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-241246ca-cdf0-4aaa-a38d-35964c0ca265 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '607fe4be34e54961a329eac3afdcced6'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:54.395922 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-241246ca-cdf0-4aaa-a38d-35964c0ca265 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '607fe4be34e54961a329eac3afdcced6'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:54.413339 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-241246ca-cdf0-4aaa-a38d-35964c0ca265 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '607fe4be34e54961a329eac3afdcced6'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:54.422270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-02312dff-bc23-41db-aac6-2c3c94abaecc None None] MFA Rules not processed for user `0d9c588e4fa34493b8490215005186ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:54.425059 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-02312dff-bc23-41db-aac6-2c3c94abaecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:54.493826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-241246ca-cdf0-4aaa-a38d-35964c0ca265 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '607fe4be34e54961a329eac3afdcced6'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:54.530999 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1085/2158] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:43:54.533990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1073/2158] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:43:53 2022] DELETE /identity/v3/users/607fe4be34e54961a329eac3afdcced6 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:43:54.534960 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42683a54-9151-49c6-8834-251a84c08668 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:54.535231 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42683a54-9151-49c6-8834-251a84c08668 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:54.535642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42683a54-9151-49c6-8834-251a84c08668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:54.540489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241246ca-cdf0-4aaa-a38d-35964c0ca265 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:54.542447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-241246ca-cdf0-4aaa-a38d-35964c0ca265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:54.572976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-42683a54-9151-49c6-8834-251a84c08668 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:54.577238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-42683a54-9151-49c6-8834-251a84c08668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:54.620807 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241246ca-cdf0-4aaa-a38d-35964c0ca265 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:54.631200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf0a6f-5fb0-44fe-b3db-90c8568c9696 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '627171bf2a2e4a21ba6f800879f8c665', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:54.634246 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf0a6f-5fb0-44fe-b3db-90c8568c9696 service glance] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:54.634796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf0a6f-5fb0-44fe-b3db-90c8568c9696 service glance] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:54.635305 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf0a6f-5fb0-44fe-b3db-90c8568c9696 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:54.715898 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1086/2159] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:54.717666 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42683a54-9151-49c6-8834-251a84c08668 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:54.718948 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf0a6f-5fb0-44fe-b3db-90c8568c9696 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:54.719503 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-42683a54-9151-49c6-8834-251a84c08668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:54.721131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf0a6f-5fb0-44fe-b3db-90c8568c9696 service glance] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:54.733799 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1074/2160] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:43:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:43:54.785711 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:54.803645 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42683a54-9151-49c6-8834-251a84c08668 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:54.808092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77424617-fbc4-47fd-a9a2-6300c1a62017 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:54.808872 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77424617-fbc4-47fd-a9a2-6300c1a62017 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:54.809258 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77424617-fbc4-47fd-a9a2-6300c1a62017 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:54.809542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77424617-fbc4-47fd-a9a2-6300c1a62017 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:54.822847 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77424617-fbc4-47fd-a9a2-6300c1a62017 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:54.824283 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77424617-fbc4-47fd-a9a2-6300c1a62017 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:54.844396 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1087/2161] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:43:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:43:54.905201 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:55.951253 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf0a6f-5fb0-44fe-b3db-90c8568c9696 service glance] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:55.952693 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-cbcf0a6f-5fb0-44fe-b3db-90c8568c9696 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:55.980471 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf0a6f-5fb0-44fe-b3db-90c8568c9696 service glance] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:55.998743 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e2d2c8-92b2-40c5-8d31-917c3f86ccd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:55.999669 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2d2c8-92b2-40c5-8d31-917c3f86ccd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:56.000288 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2d2c8-92b2-40c5-8d31-917c3f86ccd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:56.001195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2d2c8-92b2-40c5-8d31-917c3f86ccd0 admin admin] PATH_INFO: `/v3/projects/4c1d5017dd944aaf9e2d7b46e0e51185` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:56.002754 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e2d2c8-92b2-40c5-8d31-917c3f86ccd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c1d5017dd944aaf9e2d7b46e0e51185)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:56.004847 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e2d2c8-92b2-40c5-8d31-917c3f86ccd0 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:56.170396 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-54e2d2c8-92b2-40c5-8d31-917c3f86ccd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c1d5017dd944aaf9e2d7b46e0e51185', 'request_id': 'req-54e2d2c8-92b2-40c5-8d31-917c3f86ccd0'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:56.184313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-54e2d2c8-92b2-40c5-8d31-917c3f86ccd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c1d5017dd944aaf9e2d7b46e0e51185', 'request_id': 'req-54e2d2c8-92b2-40c5-8d31-917c3f86ccd0'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:56.204960 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-54e2d2c8-92b2-40c5-8d31-917c3f86ccd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c1d5017dd944aaf9e2d7b46e0e51185', 'request_id': 'req-54e2d2c8-92b2-40c5-8d31-917c3f86ccd0'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:56.293210 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-54e2d2c8-92b2-40c5-8d31-917c3f86ccd0 admin admin] The token cache is being invalidate because project 4c1d5017dd944aaf9e2d7b46e0e51185 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:43:56.293767 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-54e2d2c8-92b2-40c5-8d31-917c3f86ccd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:56.294545 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-54e2d2c8-92b2-40c5-8d31-917c3f86ccd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:43:56.296559 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1075/2162] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:43:55 2022] DELETE /identity/v3/projects/4c1d5017dd944aaf9e2d7b46e0e51185 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:56.302527 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259fbea-8b99-44a4-a2f8-01e589001417 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:56.303575 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259fbea-8b99-44a4-a2f8-01e589001417 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:56.303575 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259fbea-8b99-44a4-a2f8-01e589001417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:56.325343 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-6259fbea-8b99-44a4-a2f8-01e589001417 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:56.326563 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6259fbea-8b99-44a4-a2f8-01e589001417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:56.414777 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1088/2163] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:56.416938 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267bd57-e30a-45a0-84af-c95bac0e420c None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:56.417724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267bd57-e30a-45a0-84af-c95bac0e420c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:56.420708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267bd57-e30a-45a0-84af-c95bac0e420c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:56.451273 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-a267bd57-e30a-45a0-84af-c95bac0e420c None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:56.451273 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a267bd57-e30a-45a0-84af-c95bac0e420c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:56.457068 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1076/2164] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:56.461056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259fbea-8b99-44a4-a2f8-01e589001417 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:56.479924 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259fbea-8b99-44a4-a2f8-01e589001417 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:56.486593 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d23d6-94c6-43f1-892d-41283c02da7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:56.487606 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d23d6-94c6-43f1-892d-41283c02da7d admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:56.488108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d23d6-94c6-43f1-892d-41283c02da7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:56.488580 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d23d6-94c6-43f1-892d-41283c02da7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:56.489485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d23d6-94c6-43f1-892d-41283c02da7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:43:56.489908 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d23d6-94c6-43f1-892d-41283c02da7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:56.491670 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d23d6-94c6-43f1-892d-41283c02da7d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:56.510464 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1089/2165] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:43:56.513468 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267bd57-e30a-45a0-84af-c95bac0e420c None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:56.525288 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267bd57-e30a-45a0-84af-c95bac0e420c None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:56.530903 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca96e5e-ef74-41fb-bcfb-878d90dc0fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:56.532409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca96e5e-ef74-41fb-bcfb-878d90dc0fbf admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:56.533439 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca96e5e-ef74-41fb-bcfb-878d90dc0fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:56.537334 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca96e5e-ef74-41fb-bcfb-878d90dc0fbf admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:56.537334 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca96e5e-ef74-41fb-bcfb-878d90dc0fbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:56.537852 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca96e5e-ef74-41fb-bcfb-878d90dc0fbf admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:56.652737 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1077/2166] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:43:56 2022] POST /identity/v3/projects => generated 377 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:43:56.656597 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d23d6-94c6-43f1-892d-41283c02da7d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:56.683303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d23d6-94c6-43f1-892d-41283c02da7d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:56.718038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:43:56.782193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6124af-15c5-4d8d-b078-fd5c0462fc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:56.787952 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6124af-15c5-4d8d-b078-fd5c0462fc2f admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:56.787952 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6124af-15c5-4d8d-b078-fd5c0462fc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:56.787952 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6124af-15c5-4d8d-b078-fd5c0462fc2f admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:56.787952 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6124af-15c5-4d8d-b078-fd5c0462fc2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:56.787952 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6124af-15c5-4d8d-b078-fd5c0462fc2f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:56.895173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1090/2167] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:43:56 2022] POST /identity/v3/users => generated 289 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:43:56.900010 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca96e5e-ef74-41fb-bcfb-878d90dc0fbf admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:56.914607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca96e5e-ef74-41fb-bcfb-878d90dc0fbf admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:56.921027 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa8325-1d36-4994-94d9-15060464ead9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:56.921634 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa8325-1d36-4994-94d9-15060464ead9 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:56.921898 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa8325-1d36-4994-94d9-15060464ead9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:56.922175 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa8325-1d36-4994-94d9-15060464ead9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:56.922830 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fa8325-1d36-4994-94d9-15060464ead9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:56.924543 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fa8325-1d36-4994-94d9-15060464ead9 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:56.951702 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1078/2168] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:43:56.953682 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6124af-15c5-4d8d-b078-fd5c0462fc2f admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:56.969981 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6124af-15c5-4d8d-b078-fd5c0462fc2f admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:56.978932 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5723af-bcda-4835-b720-a4475edad52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:56.979662 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5723af-bcda-4835-b720-a4475edad52c admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:56.983062 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5723af-bcda-4835-b720-a4475edad52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:56.983062 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5723af-bcda-4835-b720-a4475edad52c admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:56.983062 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5723af-bcda-4835-b720-a4475edad52c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:56.983461 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5723af-bcda-4835-b720-a4475edad52c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:57.006056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1091/2169] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:43:57.007666 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa8325-1d36-4994-94d9-15060464ead9 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:57.030173 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa8325-1d36-4994-94d9-15060464ead9 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:57.036313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605fdcb-e3a0-4904-a0d5-8c729d9b9322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:57.037015 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605fdcb-e3a0-4904-a0d5-8c729d9b9322 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:57.037329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605fdcb-e3a0-4904-a0d5-8c729d9b9322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:57.037616 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605fdcb-e3a0-4904-a0d5-8c729d9b9322 admin admin] PATH_INFO: `/v3/projects/e87616891fa34cd9aca388a5d101f3f9/users/cd870619247a47ac810fea1aa636ac65/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:57.072896 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5605fdcb-e3a0-4904-a0d5-8c729d9b9322 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e87616891fa34cd9aca388a5d101f3f9, user_id=cd870619247a47ac810fea1aa636ac65, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:57.075740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5605fdcb-e3a0-4904-a0d5-8c729d9b9322 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:57.185488 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1079/2170] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:43:57 2022] PUT /identity/v3/projects/e87616891fa34cd9aca388a5d101f3f9/users/cd870619247a47ac810fea1aa636ac65/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:43:57.187657 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750e791-4755-4a79-8b91-a16760d8db96 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:57.188173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750e791-4755-4a79-8b91-a16760d8db96 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:57.188669 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750e791-4755-4a79-8b91-a16760d8db96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:57.231427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-9750e791-4755-4a79-8b91-a16760d8db96 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:43:57.236433 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9750e791-4755-4a79-8b91-a16760d8db96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:57.335811 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1092/2171] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:43:57.348135 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605fdcb-e3a0-4904-a0d5-8c729d9b9322 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:43:57.349887 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5605fdcb-e3a0-4904-a0d5-8c729d9b9322 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:43:57.493687 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605fdcb-e3a0-4904-a0d5-8c729d9b9322 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:43:57.511166 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f5381-bd87-4b9e-bd2a-945e036bc86d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:43:57.511803 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f5381-bd87-4b9e-bd2a-945e036bc86d service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:43:57.521340 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f5381-bd87-4b9e-bd2a-945e036bc86d service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:43:57.521340 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f5381-bd87-4b9e-bd2a-945e036bc86d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:43:57.534044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f5381-bd87-4b9e-bd2a-945e036bc86d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:43:57.535509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f5381-bd87-4b9e-bd2a-945e036bc86d service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:43:57.549449 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1080/2172] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:43:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:44:01.754237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66e90f-505d-4ce5-b832-cb4244b6114c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:44:01.754237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66e90f-505d-4ce5-b832-cb4244b6114c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:44:01.754237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66e90f-505d-4ce5-b832-cb4244b6114c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:44:01.786372 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-1b66e90f-505d-4ce5-b832-cb4244b6114c None None] MFA Rules not processed for user `00e07188d3df4d958ce83085b7bb141a`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:44:01.787607 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1b66e90f-505d-4ce5-b832-cb4244b6114c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:44:01.899072 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1093/2173] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:44:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:44:07.371890 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e9310-e033-4987-a7f6-1e985ab7369c None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:44:07.372475 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e9310-e033-4987-a7f6-1e985ab7369c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:44:07.372475 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e9310-e033-4987-a7f6-1e985ab7369c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:44:07.379699 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:44:07.408468 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-bb8e9310-e033-4987-a7f6-1e985ab7369c None None] MFA Rules not processed for user `cd870619247a47ac810fea1aa636ac65`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:44:07.410352 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-bb8e9310-e033-4987-a7f6-1e985ab7369c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:44:07.491598 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1081/2174] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:44:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:44:07.496167 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66e90f-505d-4ce5-b832-cb4244b6114c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:44:07.515130 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66e90f-505d-4ce5-b832-cb4244b6114c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:44:07.519919 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db59f38-3e26-4143-b776-273f13bf23d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:44:07.520739 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db59f38-3e26-4143-b776-273f13bf23d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:44:07.521179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db59f38-3e26-4143-b776-273f13bf23d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:44:07.521585 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db59f38-3e26-4143-b776-273f13bf23d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:44:07.522724 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db59f38-3e26-4143-b776-273f13bf23d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:44:07.524793 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db59f38-3e26-4143-b776-273f13bf23d9 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:44:07.633495 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1094/2175] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:44:07 2022] POST /identity/v3/projects => generated 379 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:44:07.637638 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e9310-e033-4987-a7f6-1e985ab7369c None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:44:07.649019 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e9310-e033-4987-a7f6-1e985ab7369c None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:44:07.713719 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37a137-ffae-4a8e-bdc5-339f5d277dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:44:07.714333 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37a137-ffae-4a8e-bdc5-339f5d277dad admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:44:07.714644 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37a137-ffae-4a8e-bdc5-339f5d277dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:44:07.714912 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37a137-ffae-4a8e-bdc5-339f5d277dad admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:44:07.715546 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37a137-ffae-4a8e-bdc5-339f5d277dad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:44:07.717580 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37a137-ffae-4a8e-bdc5-339f5d277dad admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:44:07.816951 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1082/2176] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:44:07 2022] POST /identity/v3/users => generated 296 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:44:07.819396 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db59f38-3e26-4143-b776-273f13bf23d9 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:44:07.829303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db59f38-3e26-4143-b776-273f13bf23d9 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:44:07.834648 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678ce5d-2694-407f-aafd-e6e0ab701501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:44:07.835321 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678ce5d-2694-407f-aafd-e6e0ab701501 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:44:07.835598 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678ce5d-2694-407f-aafd-e6e0ab701501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:44:07.835902 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678ce5d-2694-407f-aafd-e6e0ab701501 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:44:07.836550 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678ce5d-2694-407f-aafd-e6e0ab701501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:44:07.838190 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678ce5d-2694-407f-aafd-e6e0ab701501 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:44:07.849435 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1095/2177] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:44:07.853895 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37a137-ffae-4a8e-bdc5-339f5d277dad admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:44:07.867001 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37a137-ffae-4a8e-bdc5-339f5d277dad admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:44:07.873239 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a702a-5740-4767-8925-799a06e48401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:44:07.873977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a702a-5740-4767-8925-799a06e48401 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:44:07.874329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a702a-5740-4767-8925-799a06e48401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:44:07.874749 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a702a-5740-4767-8925-799a06e48401 admin admin] PATH_INFO: `/v3/projects/02641b8d4ca7466bb4fb396a9fdf5e11/users/fd6aaf65a27c4e95ad96508ad1288901/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:44:07.891107 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296a702a-5740-4767-8925-799a06e48401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02641b8d4ca7466bb4fb396a9fdf5e11, user_id=fd6aaf65a27c4e95ad96508ad1288901, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:44:07.894601 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296a702a-5740-4767-8925-799a06e48401 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:44:08.001977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1083/2178] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:44:07 2022] PUT /identity/v3/projects/02641b8d4ca7466bb4fb396a9fdf5e11/users/fd6aaf65a27c4e95ad96508ad1288901/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:44:08.005994 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678ce5d-2694-407f-aafd-e6e0ab701501 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:44:08.016286 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678ce5d-2694-407f-aafd-e6e0ab701501 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:44:08.021862 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:44:08.078152 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34010b-e624-419a-8199-78a1e740a0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:44:08.078774 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34010b-e624-419a-8199-78a1e740a0c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:44:08.079057 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34010b-e624-419a-8199-78a1e740a0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:44:08.079994 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34010b-e624-419a-8199-78a1e740a0c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:44:08.081136 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c34010b-e624-419a-8199-78a1e740a0c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:44:08.082842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c34010b-e624-419a-8199-78a1e740a0c0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:44:08.097949 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1096/2179] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:44:08.099762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a702a-5740-4767-8925-799a06e48401 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:44:08.125115 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a702a-5740-4767-8925-799a06e48401 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:44:08.130796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c700b-aa92-44b1-a234-be95ee63438d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:44:08.131475 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c700b-aa92-44b1-a234-be95ee63438d admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:44:08.131754 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c700b-aa92-44b1-a234-be95ee63438d admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:44:08.132028 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c700b-aa92-44b1-a234-be95ee63438d admin admin] PATH_INFO: `/v3/projects/02641b8d4ca7466bb4fb396a9fdf5e11/users/fd6aaf65a27c4e95ad96508ad1288901/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:44:08.134071 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c700b-aa92-44b1-a234-be95ee63438d admin admin] RBAC: Authorizing `identity:create_grant(project_id=02641b8d4ca7466bb4fb396a9fdf5e11, user_id=fd6aaf65a27c4e95ad96508ad1288901, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:44:08.136694 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c700b-aa92-44b1-a234-be95ee63438d admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:44:08.150516 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1084/2180] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:44:08 2022] PUT /identity/v3/projects/02641b8d4ca7466bb4fb396a9fdf5e11/users/fd6aaf65a27c4e95ad96508ad1288901/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:44:12.040085 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004bb453-f2af-4573-ba3b-731be1a8e928 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:44:12.040666 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004bb453-f2af-4573-ba3b-731be1a8e928 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:44:12.040666 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004bb453-f2af-4573-ba3b-731be1a8e928 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:44:12.042482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1097/2181] 10.222.0.33 () {54 vars in 921 bytes} [Mon Aug 8 08:44:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:44:12.049273 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db25e9-67c6-4602-8043-77139d9a902b None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:44:12.049801 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db25e9-67c6-4602-8043-77139d9a902b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:44:12.049801 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db25e9-67c6-4602-8043-77139d9a902b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:44:12.105426 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-a7db25e9-67c6-4602-8043-77139d9a902b None None] MFA Rules not processed for user `a3317f751ec74bde95ec262394001292`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:44:12.105426 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a7db25e9-67c6-4602-8043-77139d9a902b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:44:12.183733 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:44:12.226058 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1085/2182] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Aug 8 08:44:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:44:12.240366 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004bb453-f2af-4573-ba3b-731be1a8e928 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:44:12.242768 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-004bb453-f2af-4573-ba3b-731be1a8e928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:44:12.369819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004bb453-f2af-4573-ba3b-731be1a8e928 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:44:12.377253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca21eb-2cc7-4396-9902-3881b3bf3243 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '627171bf2a2e4a21ba6f800879f8c665', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:44:12.378307 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca21eb-2cc7-4396-9902-3881b3bf3243 service glance] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:44:12.378777 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca21eb-2cc7-4396-9902-3881b3bf3243 service glance] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:44:12.379213 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca21eb-2cc7-4396-9902-3881b3bf3243 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:44:12.394496 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ca21eb-2cc7-4396-9902-3881b3bf3243 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:44:12.395986 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ca21eb-2cc7-4396-9902-3881b3bf3243 service glance] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:44:12.413489 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1098/2183] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:44:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:44:12.668992 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7069a4a-00c8-41a2-98d8-4360614e5266 None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:44:12.671454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7069a4a-00c8-41a2-98d8-4360614e5266 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:44:12.672691 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7069a4a-00c8-41a2-98d8-4360614e5266 None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:44:12.676243 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1086/2184] 10.222.0.33 () {54 vars in 921 bytes} [Mon Aug 8 08:44:12 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:44:12.682593 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b167fa-3b9b-465e-802b-d9791f65d17a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:44:12.683115 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b167fa-3b9b-465e-802b-d9791f65d17a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:44:12.683392 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b167fa-3b9b-465e-802b-d9791f65d17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:44:12.713921 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-57b167fa-3b9b-465e-802b-d9791f65d17a None None] MFA Rules not processed for user `2c3b8b43026c40209bfc42acd2830610`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:44:12.725789 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-57b167fa-3b9b-465e-802b-d9791f65d17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:44:12.808382 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1099/2185] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Aug 8 08:44:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:44:12.838140 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7069a4a-00c8-41a2-98d8-4360614e5266 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:44:12.863534 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7069a4a-00c8-41a2-98d8-4360614e5266 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:44:12.879449 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11792ff-bffb-4d99-b2f0-ca54954bd007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:44:12.883681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11792ff-bffb-4d99-b2f0-ca54954bd007 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:44:12.883681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11792ff-bffb-4d99-b2f0-ca54954bd007 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:44:12.883681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11792ff-bffb-4d99-b2f0-ca54954bd007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:44:12.912350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11792ff-bffb-4d99-b2f0-ca54954bd007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:44:12.915574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11792ff-bffb-4d99-b2f0-ca54954bd007 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:44:12.946046 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1087/2186] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:44:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:44:13.121720 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092c667-5f20-4e98-a882-9341d03ec5cd None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:44:13.125557 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092c667-5f20-4e98-a882-9341d03ec5cd None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:44:13.125557 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092c667-5f20-4e98-a882-9341d03ec5cd None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:44:13.125557 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1100/2187] 10.222.0.33 () {54 vars in 921 bytes} [Mon Aug 8 08:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:44:13.131590 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384e36c-80f3-4109-8d33-71d7d394c380 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:44:13.132052 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384e36c-80f3-4109-8d33-71d7d394c380 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:44:13.132230 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384e36c-80f3-4109-8d33-71d7d394c380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:44:13.155245 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-0384e36c-80f3-4109-8d33-71d7d394c380 None None] MFA Rules not processed for user `a3317f751ec74bde95ec262394001292`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:44:13.157229 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0384e36c-80f3-4109-8d33-71d7d394c380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:44:13.169167 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1088/2188] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Aug 8 08:44:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:44:13.182967 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092c667-5f20-4e98-a882-9341d03ec5cd None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:44:13.201698 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092c667-5f20-4e98-a882-9341d03ec5cd None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:44:13.209008 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea20695-6926-4ea0-9910-55730e43f352 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:44:13.209992 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea20695-6926-4ea0-9910-55730e43f352 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:44:13.210420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea20695-6926-4ea0-9910-55730e43f352 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:44:13.210838 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea20695-6926-4ea0-9910-55730e43f352 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:44:13.222599 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea20695-6926-4ea0-9910-55730e43f352 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:44:13.225084 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea20695-6926-4ea0-9910-55730e43f352 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:44:13.245680 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1101/2189] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:44:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:44:15.339375 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b065d1-4d3b-4bea-9d53-1e45805940c9 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:44:15.339878 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b065d1-4d3b-4bea-9d53-1e45805940c9 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:44:15.340045 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b065d1-4d3b-4bea-9d53-1e45805940c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:44:15.375656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-51b065d1-4d3b-4bea-9d53-1e45805940c9 None None] MFA Rules not processed for user `fd6aaf65a27c4e95ad96508ad1288901`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:44:15.378663 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-51b065d1-4d3b-4bea-9d53-1e45805940c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:44:15.476120 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1089/2190] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:44:15 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:44:15.485312 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea20695-6926-4ea0-9910-55730e43f352 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:44:15.486850 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8ea20695-6926-4ea0-9910-55730e43f352 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:44:15.527181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea20695-6926-4ea0-9910-55730e43f352 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:44:15.540319 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e14e3-3b63-4e44-b846-c693fdc6ea04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:44:15.541204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e14e3-3b63-4e44-b846-c693fdc6ea04 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:44:15.541489 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e14e3-3b63-4e44-b846-c693fdc6ea04 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:44:15.541757 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e14e3-3b63-4e44-b846-c693fdc6ea04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:44:15.607204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:44:15.625696 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e14e3-3b63-4e44-b846-c693fdc6ea04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:44:15.627730 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e14e3-3b63-4e44-b846-c693fdc6ea04 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:44:15.639297 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1102/2191] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:44:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:44:20.840141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d57659-dc1a-412b-b7d7-a10f302d888f None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:44:20.840704 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d57659-dc1a-412b-b7d7-a10f302d888f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:44:20.840819 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d57659-dc1a-412b-b7d7-a10f302d888f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:44:20.861565 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-35d57659-dc1a-412b-b7d7-a10f302d888f None None] MFA Rules not processed for user `7f3422c6f9a041ceba851b71bd15d0df`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:44:20.862854 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-35d57659-dc1a-412b-b7d7-a10f302d888f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:44:20.916662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1090/2192] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 08:44:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:44:20.929063 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e14e3-3b63-4e44-b846-c693fdc6ea04 service nova] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:44:20.930494 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ff6e14e3-3b63-4e44-b846-c693fdc6ea04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:44:21.001971 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e14e3-3b63-4e44-b846-c693fdc6ea04 service nova] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:44:21.007713 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a48beb-0008-4276-91d1-f9450458faf7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e69c0e02ad944f2a85fc5318018e5cb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:44:21.008333 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a48beb-0008-4276-91d1-f9450458faf7 service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:44:21.008620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a48beb-0008-4276-91d1-f9450458faf7 service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:44:21.008960 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a48beb-0008-4276-91d1-f9450458faf7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:44:21.018652 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a48beb-0008-4276-91d1-f9450458faf7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:44:21.020267 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a48beb-0008-4276-91d1-f9450458faf7 service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:44:21.028513 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1103/2193] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 08:44:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 08:44:25.615617 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d57659-dc1a-412b-b7d7-a10f302d888f None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:44:25.617989 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-35d57659-dc1a-412b-b7d7-a10f302d888f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:44:25.692303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:44:25.729431 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d57659-dc1a-412b-b7d7-a10f302d888f None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:44:25.735722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28b6de-2c85-44a4-8b5a-9ee357f333f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:44:25.736999 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28b6de-2c85-44a4-8b5a-9ee357f333f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:44:25.737401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28b6de-2c85-44a4-8b5a-9ee357f333f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:44:25.737876 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28b6de-2c85-44a4-8b5a-9ee357f333f0 admin admin] PATH_INFO: `/v3/users/0d9c588e4fa34493b8490215005186ee` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:44:25.739681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28b6de-2c85-44a4-8b5a-9ee357f333f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d9c588e4fa34493b8490215005186ee)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:44:25.746567 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28b6de-2c85-44a4-8b5a-9ee357f333f0 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:44:25.949880 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ea28b6de-2c85-44a4-8b5a-9ee357f333f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d9c588e4fa34493b8490215005186ee'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:44:25.950676 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ea28b6de-2c85-44a4-8b5a-9ee357f333f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d9c588e4fa34493b8490215005186ee'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:44:25.951327 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ea28b6de-2c85-44a4-8b5a-9ee357f333f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d9c588e4fa34493b8490215005186ee'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:44:25.959521 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ea28b6de-2c85-44a4-8b5a-9ee357f333f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d9c588e4fa34493b8490215005186ee'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:44:26.088286 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ea28b6de-2c85-44a4-8b5a-9ee357f333f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d9c588e4fa34493b8490215005186ee'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:44:26.107044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1091/2194] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:44:25 2022] DELETE /identity/v3/users/0d9c588e4fa34493b8490215005186ee => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:44:26.111846 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123dde2-9020-4fba-b0fb-b1c8b3d33f94 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:44:26.112427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123dde2-9020-4fba-b0fb-b1c8b3d33f94 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:44:26.112427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123dde2-9020-4fba-b0fb-b1c8b3d33f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:44:26.131271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-f123dde2-9020-4fba-b0fb-b1c8b3d33f94 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:44:26.132650 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f123dde2-9020-4fba-b0fb-b1c8b3d33f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:44:26.204024 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1104/2195] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:44:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:44:26.216365 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28b6de-2c85-44a4-8b5a-9ee357f333f0 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:44:26.216843 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ea28b6de-2c85-44a4-8b5a-9ee357f333f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:44:26.278613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28b6de-2c85-44a4-8b5a-9ee357f333f0 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:44:26.282347 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258e121-e698-40a3-b734-0fb21693f1a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:44:26.282965 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258e121-e698-40a3-b734-0fb21693f1a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:44:26.283766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258e121-e698-40a3-b734-0fb21693f1a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:44:26.284137 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258e121-e698-40a3-b734-0fb21693f1a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:44:26.294174 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a258e121-e698-40a3-b734-0fb21693f1a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:44:26.295928 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a258e121-e698-40a3-b734-0fb21693f1a0 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:44:26.307716 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1092/2196] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:44:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:44:26.755565 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f123dde2-9020-4fba-b0fb-b1c8b3d33f94 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:44:26.755565 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f123dde2-9020-4fba-b0fb-b1c8b3d33f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:44:26.768574 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f123dde2-9020-4fba-b0fb-b1c8b3d33f94 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:44:26.773985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c2f606-6285-49c5-bdac-26b426658c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:44:26.774737 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2f606-6285-49c5-bdac-26b426658c96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:44:26.775112 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2f606-6285-49c5-bdac-26b426658c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:44:26.775485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2f606-6285-49c5-bdac-26b426658c96 admin admin] PATH_INFO: `/v3/projects/2f9b093fc72d42798e6e3c3ea42516f2` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:44:26.776840 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c2f606-6285-49c5-bdac-26b426658c96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f9b093fc72d42798e6e3c3ea42516f2)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:44:26.779011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c2f606-6285-49c5-bdac-26b426658c96 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:44:26.896370 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f2c2f606-6285-49c5-bdac-26b426658c96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f9b093fc72d42798e6e3c3ea42516f2', 'request_id': 'req-f2c2f606-6285-49c5-bdac-26b426658c96'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:44:26.969930 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f2c2f606-6285-49c5-bdac-26b426658c96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f9b093fc72d42798e6e3c3ea42516f2', 'request_id': 'req-f2c2f606-6285-49c5-bdac-26b426658c96'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:44:26.980811 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f2c2f606-6285-49c5-bdac-26b426658c96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f9b093fc72d42798e6e3c3ea42516f2', 'request_id': 'req-f2c2f606-6285-49c5-bdac-26b426658c96'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:44:26.991963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:44:27.003621 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f2c2f606-6285-49c5-bdac-26b426658c96 admin admin] The token cache is being invalidate because project 2f9b093fc72d42798e6e3c3ea42516f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:44:27.003878 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f2c2f606-6285-49c5-bdac-26b426658c96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:44:27.004599 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f2c2f606-6285-49c5-bdac-26b426658c96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:44:27.006412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1105/2197] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:44:26 2022] DELETE /identity/v3/projects/2f9b093fc72d42798e6e3c3ea42516f2 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:44:36.302287 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=58,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:44:37.133735 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:44:37.327711 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d03af9-92f3-40d0-9ce6-bed6e3d8b33f None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:44:37.328192 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d03af9-92f3-40d0-9ce6-bed6e3d8b33f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:44:37.328284 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d03af9-92f3-40d0-9ce6-bed6e3d8b33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:44:37.350197 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-26d03af9-92f3-40d0-9ce6-bed6e3d8b33f None None] MFA Rules not processed for user `cd870619247a47ac810fea1aa636ac65`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:44:37.352073 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-26d03af9-92f3-40d0-9ce6-bed6e3d8b33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:44:37.422376 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1093/2198] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:44:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:44:44.308321 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c2f606-6285-49c5-bdac-26b426658c96 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:44:44.309764 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f2c2f606-6285-49c5-bdac-26b426658c96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:44:44.383736 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c2f606-6285-49c5-bdac-26b426658c96 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:44:44.388694 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579d896-dc57-4f8e-89ad-57d5c33e628e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c23a782b9b4144bace1248c3b2e754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:44:44.389356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579d896-dc57-4f8e-89ad-57d5c33e628e service placement] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:44:44.389636 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579d896-dc57-4f8e-89ad-57d5c33e628e service placement] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:44:44.389908 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579d896-dc57-4f8e-89ad-57d5c33e628e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:44:44.391695 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e579d896-dc57-4f8e-89ad-57d5c33e628e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:44:44.404048 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e579d896-dc57-4f8e-89ad-57d5c33e628e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:44:44.405599 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e579d896-dc57-4f8e-89ad-57d5c33e628e service placement] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:44:44.418638 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1106/2199] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:44:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:44:47.403762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=25 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:44:51.142085 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce6a06-abd9-4416-80db-71f8988f52d5 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:44:51.142768 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce6a06-abd9-4416-80db-71f8988f52d5 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:44:51.142768 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce6a06-abd9-4416-80db-71f8988f52d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:44:51.165763 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-0dce6a06-abd9-4416-80db-71f8988f52d5 None None] MFA Rules not processed for user `7f3422c6f9a041ceba851b71bd15d0df`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:44:51.166973 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0dce6a06-abd9-4416-80db-71f8988f52d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:44:51.234383 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1094/2200] 10.222.0.33 () {58 vars in 967 bytes} [Mon Aug 8 08:44:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:44:51.279075 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579d896-dc57-4f8e-89ad-57d5c33e628e service placement] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:44:51.279075 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e579d896-dc57-4f8e-89ad-57d5c33e628e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:44:51.352298 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579d896-dc57-4f8e-89ad-57d5c33e628e service placement] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:44:51.357868 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3c606-9042-4651-aacd-ca8e2143586d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e69c0e02ad944f2a85fc5318018e5cb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:44:51.358478 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3c606-9042-4651-aacd-ca8e2143586d service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:44:51.358757 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3c606-9042-4651-aacd-ca8e2143586d service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:44:51.359026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3c606-9042-4651-aacd-ca8e2143586d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:44:51.367638 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e3c606-9042-4651-aacd-ca8e2143586d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:44:51.369260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e3c606-9042-4651-aacd-ca8e2143586d service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:44:51.377820 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1107/2201] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 08:44:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 08:45:01.225056 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=25 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:45:01.374405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=64 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:45:05.045805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce6a06-abd9-4416-80db-71f8988f52d5 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:45:05.047454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0dce6a06-abd9-4416-80db-71f8988f52d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:45:05.105251 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce6a06-abd9-4416-80db-71f8988f52d5 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:45:05.109084 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b0a8e-dde3-465a-94c3-4d737ff4d458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:45:05.113358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b0a8e-dde3-465a-94c3-4d737ff4d458 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:45:05.113358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b0a8e-dde3-465a-94c3-4d737ff4d458 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:45:05.113358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b0a8e-dde3-465a-94c3-4d737ff4d458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:45:05.113358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d86b0a8e-dde3-465a-94c3-4d737ff4d458 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:45:05.179029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b0a8e-dde3-465a-94c3-4d737ff4d458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:45:05.180481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b0a8e-dde3-465a-94c3-4d737ff4d458 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:45:05.192237 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1095/2202] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:45:05 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:45:05.349433 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a199f-b005-4f83-bb85-e6e4f0956da2 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:45:05.349433 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a199f-b005-4f83-bb85-e6e4f0956da2 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:45:05.350137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a199f-b005-4f83-bb85-e6e4f0956da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:45:05.370935 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-1f0a199f-b005-4f83-bb85-e6e4f0956da2 None None] MFA Rules not processed for user `32dcdc2e24e44a07ade44d1da330a77e`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:45:05.372272 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1f0a199f-b005-4f83-bb85-e6e4f0956da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:45:05.434224 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1108/2203] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:45:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:45:15.186831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=50 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:45:15.425831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=25 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:45:44.890013 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d535e-e334-4e49-a6dc-2b6168464347 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:45:44.890546 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d535e-e334-4e49-a6dc-2b6168464347 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:45:44.890644 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d535e-e334-4e49-a6dc-2b6168464347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:45:44.910413 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-c03d535e-e334-4e49-a6dc-2b6168464347 None None] MFA Rules not processed for user `00e07188d3df4d958ce83085b7bb141a`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:45:44.911605 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c03d535e-e334-4e49-a6dc-2b6168464347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:45:44.967235 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1096/2204] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:45:44 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:45:46.913472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b959c-9fbb-44f4-80bc-48a82a9192f9 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:45:46.914038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b959c-9fbb-44f4-80bc-48a82a9192f9 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:45:46.914038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b959c-9fbb-44f4-80bc-48a82a9192f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:45:46.932642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-5d8b959c-9fbb-44f4-80bc-48a82a9192f9 None None] MFA Rules not processed for user `b3e5a78c3b6c48d9a4aed7d36fd54736`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:45:46.933854 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5d8b959c-9fbb-44f4-80bc-48a82a9192f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:45:46.989679 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1109/2205] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:45:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:45:54.959958 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=25 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:45:56.982790 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=25 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:46:24.017739 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2893a-aa5b-41ce-a4d7-928e4d4c2195 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:46:24.018286 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2893a-aa5b-41ce-a4d7-928e4d4c2195 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:46:24.018286 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2893a-aa5b-41ce-a4d7-928e4d4c2195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:46:24.037673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-abe2893a-aa5b-41ce-a4d7-928e4d4c2195 None None] MFA Rules not processed for user `32dcdc2e24e44a07ade44d1da330a77e`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:46:24.039018 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-abe2893a-aa5b-41ce-a4d7-928e4d4c2195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:46:24.046458 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1097/2206] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:46:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:46:30.779071 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b959c-9fbb-44f4-80bc-48a82a9192f9 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:46:30.786045 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c62078-c020-452d-9403-9beb8ce69342 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:46:30.786548 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c62078-c020-452d-9403-9beb8ce69342 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:46:30.786668 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c62078-c020-452d-9403-9beb8ce69342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:46:30.790812 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b959c-9fbb-44f4-80bc-48a82a9192f9 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:46:30.795558 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae046e-fa8c-4fd3-939f-22d34d639e30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:46:30.807986 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae046e-fa8c-4fd3-939f-22d34d639e30 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:46:30.807986 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae046e-fa8c-4fd3-939f-22d34d639e30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:46:30.807986 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae046e-fa8c-4fd3-939f-22d34d639e30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:46:30.807986 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f7ae046e-fa8c-4fd3-939f-22d34d639e30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:46:30.808830 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-f0c62078-c020-452d-9403-9beb8ce69342 None None] MFA Rules not processed for user `81d6a9d89e394e63ba5c8182a12b4142`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:46:30.810642 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f0c62078-c020-452d-9403-9beb8ce69342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:46:30.883781 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1098/2207] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:46:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:46:30.890836 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae046e-fa8c-4fd3-939f-22d34d639e30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:46:30.892294 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae046e-fa8c-4fd3-939f-22d34d639e30 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:46:30.904185 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1110/2208] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:46:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:46:31.256165 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c62078-c020-452d-9403-9beb8ce69342 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:46:31.266604 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c62078-c020-452d-9403-9beb8ce69342 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:46:31.269445 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b936e17-54bd-4dca-9626-ad1828885901 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:46:31.270044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b936e17-54bd-4dca-9626-ad1828885901 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:46:31.270320 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b936e17-54bd-4dca-9626-ad1828885901 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:46:31.270590 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b936e17-54bd-4dca-9626-ad1828885901 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:46:31.274656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8b936e17-54bd-4dca-9626-ad1828885901 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:46:31.288187 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b936e17-54bd-4dca-9626-ad1828885901 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:46:31.289772 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b936e17-54bd-4dca-9626-ad1828885901 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:46:31.302171 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1099/2209] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:46:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:46:39.396090 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae046e-fa8c-4fd3-939f-22d34d639e30 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:46:39.397678 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f7ae046e-fa8c-4fd3-939f-22d34d639e30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:46:39.477030 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae046e-fa8c-4fd3-939f-22d34d639e30 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:46:39.482572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4816c661-a590-4d3c-b7ff-2eab84b576b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:46:39.483379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816c661-a590-4d3c-b7ff-2eab84b576b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:46:39.483819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816c661-a590-4d3c-b7ff-2eab84b576b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:46:39.484403 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816c661-a590-4d3c-b7ff-2eab84b576b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:46:39.486348 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4816c661-a590-4d3c-b7ff-2eab84b576b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:46:39.512131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4816c661-a590-4d3c-b7ff-2eab84b576b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:46:39.514020 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4816c661-a590-4d3c-b7ff-2eab84b576b3 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:46:39.525931 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1111/2210] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:46:39 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:46:41.297495 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=34 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:46:47.938182 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b936e17-54bd-4dca-9626-ad1828885901 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:46:47.939650 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8b936e17-54bd-4dca-9626-ad1828885901 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:46:47.953756 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b936e17-54bd-4dca-9626-ad1828885901 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:46:47.959271 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b9a3ed-7775-438d-a598-49404a77133a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:46:47.959859 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9a3ed-7775-438d-a598-49404a77133a service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:46:47.960141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9a3ed-7775-438d-a598-49404a77133a service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:46:47.960417 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9a3ed-7775-438d-a598-49404a77133a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:46:47.962120 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-49b9a3ed-7775-438d-a598-49404a77133a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:46:47.973131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b9a3ed-7775-438d-a598-49404a77133a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:46:47.974878 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b9a3ed-7775-438d-a598-49404a77133a service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:46:47.984830 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1100/2211] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:46:47 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:46:48.012010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4816c661-a590-4d3c-b7ff-2eab84b576b3 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:46:48.013528 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4816c661-a590-4d3c-b7ff-2eab84b576b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:46:48.029446 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4816c661-a590-4d3c-b7ff-2eab84b576b3 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:46:48.035006 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c7f12-cb3e-49a0-a754-f1c8d1e073f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:46:48.035601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c7f12-cb3e-49a0-a754-f1c8d1e073f0 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:46:48.035879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c7f12-cb3e-49a0-a754-f1c8d1e073f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:46:48.036149 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c7f12-cb3e-49a0-a754-f1c8d1e073f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:46:48.037956 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b77c7f12-cb3e-49a0-a754-f1c8d1e073f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:46:48.049637 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77c7f12-cb3e-49a0-a754-f1c8d1e073f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:46:48.051263 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77c7f12-cb3e-49a0-a754-f1c8d1e073f0 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:46:48.061104 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1112/2212] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:46:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:46:54.040207 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b9a3ed-7775-438d-a598-49404a77133a service nova] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:46:54.049514 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b9a3ed-7775-438d-a598-49404a77133a service nova] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:46:54.052991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f60a58b-4a74-4f3a-8cb1-e9634d51583c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:46:54.053607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60a58b-4a74-4f3a-8cb1-e9634d51583c service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:46:54.053886 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60a58b-4a74-4f3a-8cb1-e9634d51583c service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:46:54.054158 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60a58b-4a74-4f3a-8cb1-e9634d51583c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:46:54.055890 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0f60a58b-4a74-4f3a-8cb1-e9634d51583c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:46:54.117940 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f60a58b-4a74-4f3a-8cb1-e9634d51583c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:46:54.119355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f60a58b-4a74-4f3a-8cb1-e9634d51583c service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:46:54.129638 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1101/2213] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:46:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:46:57.658580 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c7f12-cb3e-49a0-a754-f1c8d1e073f0 service nova] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:46:57.664215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=67 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:46:57.671330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c7f12-cb3e-49a0-a754-f1c8d1e073f0 service nova] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:46:57.676779 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31bf87-3f50-482b-a271-ae5dedb0ef82 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c23a782b9b4144bace1248c3b2e754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:46:57.677421 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31bf87-3f50-482b-a271-ae5dedb0ef82 service placement] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:46:57.677702 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31bf87-3f50-482b-a271-ae5dedb0ef82 service placement] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:46:57.677973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31bf87-3f50-482b-a271-ae5dedb0ef82 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:46:57.679904 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bc31bf87-3f50-482b-a271-ae5dedb0ef82 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:46:57.692574 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc31bf87-3f50-482b-a271-ae5dedb0ef82 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:46:57.694196 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc31bf87-3f50-482b-a271-ae5dedb0ef82 service placement] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:46:57.704918 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1113/2214] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:46:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:46:57.784147 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f60a58b-4a74-4f3a-8cb1-e9634d51583c service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:46:57.785669 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0f60a58b-4a74-4f3a-8cb1-e9634d51583c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:46:57.799191 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f60a58b-4a74-4f3a-8cb1-e9634d51583c service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:46:57.804112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215eaff-9d35-457b-bf1a-fca9f2cd563b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:46:57.804743 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215eaff-9d35-457b-bf1a-fca9f2cd563b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:46:57.805023 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215eaff-9d35-457b-bf1a-fca9f2cd563b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:46:57.805331 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215eaff-9d35-457b-bf1a-fca9f2cd563b admin admin] PATH_INFO: `/v3/users/00e07188d3df4d958ce83085b7bb141a` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:46:57.806555 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1215eaff-9d35-457b-bf1a-fca9f2cd563b admin admin] RBAC: Authorizing `identity:delete_user(user_id=00e07188d3df4d958ce83085b7bb141a)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:46:57.808489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1215eaff-9d35-457b-bf1a-fca9f2cd563b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:46:57.956250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1215eaff-9d35-457b-bf1a-fca9f2cd563b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00e07188d3df4d958ce83085b7bb141a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:46:57.956903 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1215eaff-9d35-457b-bf1a-fca9f2cd563b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00e07188d3df4d958ce83085b7bb141a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:46:57.957560 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1215eaff-9d35-457b-bf1a-fca9f2cd563b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00e07188d3df4d958ce83085b7bb141a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:46:57.965378 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1215eaff-9d35-457b-bf1a-fca9f2cd563b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00e07188d3df4d958ce83085b7bb141a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:46:58.016434 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1215eaff-9d35-457b-bf1a-fca9f2cd563b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00e07188d3df4d958ce83085b7bb141a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:46:58.035526 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1102/2215] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:46:57 2022] DELETE /identity/v3/users/00e07188d3df4d958ce83085b7bb141a => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:46:58.271141 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31bf87-3f50-482b-a271-ae5dedb0ef82 service placement] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:46:58.286664 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31bf87-3f50-482b-a271-ae5dedb0ef82 service placement] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:46:58.362327 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d3936-1617-490a-a3a2-78bfd10d2867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:46:58.363031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d3936-1617-490a-a3a2-78bfd10d2867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:46:58.363318 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d3936-1617-490a-a3a2-78bfd10d2867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:46:58.363635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d3936-1617-490a-a3a2-78bfd10d2867 admin admin] PATH_INFO: `/v3/projects/2a4e5c5a950c4047b15749aace25f46d` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:46:58.365380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d3936-1617-490a-a3a2-78bfd10d2867 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a4e5c5a950c4047b15749aace25f46d)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:46:58.368080 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d3936-1617-490a-a3a2-78bfd10d2867 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:46:58.486262 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-201d3936-1617-490a-a3a2-78bfd10d2867 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a4e5c5a950c4047b15749aace25f46d', 'request_id': 'req-201d3936-1617-490a-a3a2-78bfd10d2867'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:46:58.530589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-201d3936-1617-490a-a3a2-78bfd10d2867 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a4e5c5a950c4047b15749aace25f46d', 'request_id': 'req-201d3936-1617-490a-a3a2-78bfd10d2867'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:46:58.537022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-201d3936-1617-490a-a3a2-78bfd10d2867 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a4e5c5a950c4047b15749aace25f46d', 'request_id': 'req-201d3936-1617-490a-a3a2-78bfd10d2867'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:46:58.548049 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-201d3936-1617-490a-a3a2-78bfd10d2867 admin admin] The token cache is being invalidate because project 2a4e5c5a950c4047b15749aace25f46d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:46:58.548331 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-201d3936-1617-490a-a3a2-78bfd10d2867 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:46:58.548957 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-201d3936-1617-490a-a3a2-78bfd10d2867 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:46:58.550453 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1114/2216] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:46:58 2022] DELETE /identity/v3/projects/2a4e5c5a950c4047b15749aace25f46d => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:46:58.555434 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95953944-f690-4263-9cf6-5f13b9a7733b None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:46:58.560228 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95953944-f690-4263-9cf6-5f13b9a7733b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:46:58.560228 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95953944-f690-4263-9cf6-5f13b9a7733b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:46:58.575888 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-95953944-f690-4263-9cf6-5f13b9a7733b None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:46:58.577103 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-95953944-f690-4263-9cf6-5f13b9a7733b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:46:58.623534 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:46:58.635194 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1103/2217] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:46:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:46:58.642936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3a640-f650-42c7-93c2-f088dcc34d01 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:46:58.643219 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3a640-f650-42c7-93c2-f088dcc34d01 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:46:58.643488 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3a640-f650-42c7-93c2-f088dcc34d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:46:58.663270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e9a3a640-f650-42c7-93c2-f088dcc34d01 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:46:58.664396 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e9a3a640-f650-42c7-93c2-f088dcc34d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:46:58.672813 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1115/2218] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:46:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:46:58.676944 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95953944-f690-4263-9cf6-5f13b9a7733b None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:46:58.686870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95953944-f690-4263-9cf6-5f13b9a7733b None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:46:58.691302 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89678650-6389-4a11-a4f6-e098eac0f290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:46:58.691891 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89678650-6389-4a11-a4f6-e098eac0f290 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:46:58.694636 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89678650-6389-4a11-a4f6-e098eac0f290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:46:58.694636 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89678650-6389-4a11-a4f6-e098eac0f290 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:46:58.694636 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89678650-6389-4a11-a4f6-e098eac0f290 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:46:58.694636 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89678650-6389-4a11-a4f6-e098eac0f290 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:46:58.695022 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89678650-6389-4a11-a4f6-e098eac0f290 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:46:58.706278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1104/2219] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:46:58.710454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3a640-f650-42c7-93c2-f088dcc34d01 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:46:58.720313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3a640-f650-42c7-93c2-f088dcc34d01 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:46:58.724912 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a4615-d652-461a-b049-8b56a8bc74c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:46:58.725523 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a4615-d652-461a-b049-8b56a8bc74c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:46:58.725798 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a4615-d652-461a-b049-8b56a8bc74c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:46:58.726063 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a4615-d652-461a-b049-8b56a8bc74c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:46:58.726681 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632a4615-d652-461a-b049-8b56a8bc74c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:46:58.728551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632a4615-d652-461a-b049-8b56a8bc74c9 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:46:58.830624 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1116/2220] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:46:58 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:46:58.834805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89678650-6389-4a11-a4f6-e098eac0f290 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:46:58.845339 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89678650-6389-4a11-a4f6-e098eac0f290 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:46:58.899082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32098203-4f84-43ca-a4d0-a3f7f92c458b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:46:58.899716 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32098203-4f84-43ca-a4d0-a3f7f92c458b admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:46:58.899996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32098203-4f84-43ca-a4d0-a3f7f92c458b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:46:58.900268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32098203-4f84-43ca-a4d0-a3f7f92c458b admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:46:58.900884 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32098203-4f84-43ca-a4d0-a3f7f92c458b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:46:58.903508 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32098203-4f84-43ca-a4d0-a3f7f92c458b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:46:58.993615 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1105/2221] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:46:58 2022] POST /identity/v3/users => generated 288 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:46:58.997911 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a4615-d652-461a-b049-8b56a8bc74c9 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:46:59.008210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a4615-d652-461a-b049-8b56a8bc74c9 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:46:59.013527 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb51f7d-72bd-400e-bcc3-326095f8633d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:46:59.014123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb51f7d-72bd-400e-bcc3-326095f8633d admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:46:59.014402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb51f7d-72bd-400e-bcc3-326095f8633d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:46:59.014673 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb51f7d-72bd-400e-bcc3-326095f8633d admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:46:59.015317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb51f7d-72bd-400e-bcc3-326095f8633d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:46:59.016894 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb51f7d-72bd-400e-bcc3-326095f8633d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:46:59.026748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1117/2222] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:46:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:46:59.031356 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32098203-4f84-43ca-a4d0-a3f7f92c458b admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:46:59.041422 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32098203-4f84-43ca-a4d0-a3f7f92c458b admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:46:59.046276 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f4c516-5b66-4310-8887-6ad2f3f1aba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:46:59.046889 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4c516-5b66-4310-8887-6ad2f3f1aba3 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:46:59.047166 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4c516-5b66-4310-8887-6ad2f3f1aba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:46:59.047436 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4c516-5b66-4310-8887-6ad2f3f1aba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:46:59.048067 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f4c516-5b66-4310-8887-6ad2f3f1aba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:46:59.049775 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f4c516-5b66-4310-8887-6ad2f3f1aba3 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:46:59.060468 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1106/2223] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:46:59.064550 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb51f7d-72bd-400e-bcc3-326095f8633d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:46:59.075083 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb51f7d-72bd-400e-bcc3-326095f8633d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:46:59.079900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936399f0-30d4-4eee-89f6-c66c32fb4d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:46:59.080547 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936399f0-30d4-4eee-89f6-c66c32fb4d60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:46:59.080824 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936399f0-30d4-4eee-89f6-c66c32fb4d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:46:59.081099 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936399f0-30d4-4eee-89f6-c66c32fb4d60 admin admin] PATH_INFO: `/v3/projects/a263438c78bf4c34be8f427c12d8a291/users/cccd76799a5140bc925f01e63abf45e2/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:46:59.095807 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936399f0-30d4-4eee-89f6-c66c32fb4d60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a263438c78bf4c34be8f427c12d8a291, user_id=cccd76799a5140bc925f01e63abf45e2, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:46:59.098522 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936399f0-30d4-4eee-89f6-c66c32fb4d60 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:46:59.188793 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1118/2224] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:46:59 2022] PUT /identity/v3/projects/a263438c78bf4c34be8f427c12d8a291/users/cccd76799a5140bc925f01e63abf45e2/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:46:59.198902 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff49c19-d2ae-4087-bf4b-7a9fce8b86a6 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:46:59.199816 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff49c19-d2ae-4087-bf4b-7a9fce8b86a6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:46:59.200203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff49c19-d2ae-4087-bf4b-7a9fce8b86a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:46:59.229414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-9ff49c19-d2ae-4087-bf4b-7a9fce8b86a6 None None] MFA Rules not processed for user `cccd76799a5140bc925f01e63abf45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:46:59.230632 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-9ff49c19-d2ae-4087-bf4b-7a9fce8b86a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:46:59.296373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1107/2225] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:46:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:46:59.307716 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936399f0-30d4-4eee-89f6-c66c32fb4d60 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:46:59.310059 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-936399f0-30d4-4eee-89f6-c66c32fb4d60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:46:59.333846 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:46:59.385121 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936399f0-30d4-4eee-89f6-c66c32fb4d60 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:46:59.391319 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8debcf95-ad15-4afd-9b3a-08d709341941 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:46:59.391923 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8debcf95-ad15-4afd-9b3a-08d709341941 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:46:59.392205 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8debcf95-ad15-4afd-9b3a-08d709341941 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:46:59.392479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8debcf95-ad15-4afd-9b3a-08d709341941 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:46:59.400463 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8debcf95-ad15-4afd-9b3a-08d709341941 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:46:59.402094 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8debcf95-ad15-4afd-9b3a-08d709341941 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:46:59.411461 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1119/2226] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:46:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:46:59.679356 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff49c19-d2ae-4087-bf4b-7a9fce8b86a6 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:46:59.729389 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:46:59.743560 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff49c19-d2ae-4087-bf4b-7a9fce8b86a6 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:46:59.748412 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de56b9-44f4-48cc-93c1-10b27c97e619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:46:59.749040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de56b9-44f4-48cc-93c1-10b27c97e619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:46:59.749373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de56b9-44f4-48cc-93c1-10b27c97e619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:46:59.749649 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de56b9-44f4-48cc-93c1-10b27c97e619 admin admin] PATH_INFO: `/v3/users/cccd76799a5140bc925f01e63abf45e2` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:46:59.750763 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10de56b9-44f4-48cc-93c1-10b27c97e619 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cccd76799a5140bc925f01e63abf45e2)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:46:59.752732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10de56b9-44f4-48cc-93c1-10b27c97e619 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:46:59.787014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486da08-435e-4544-8a69-2e601de2349e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:46:59.787506 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486da08-435e-4544-8a69-2e601de2349e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:46:59.787612 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486da08-435e-4544-8a69-2e601de2349e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:46:59.806428 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-6486da08-435e-4544-8a69-2e601de2349e None None] MFA Rules not processed for user `7f3422c6f9a041ceba851b71bd15d0df`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:46:59.807538 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6486da08-435e-4544-8a69-2e601de2349e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:46:59.863678 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1120/2227] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 08:46:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:46:59.902383 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-10de56b9-44f4-48cc-93c1-10b27c97e619 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cccd76799a5140bc925f01e63abf45e2'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:46:59.903905 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-10de56b9-44f4-48cc-93c1-10b27c97e619 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cccd76799a5140bc925f01e63abf45e2'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:46:59.904512 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-10de56b9-44f4-48cc-93c1-10b27c97e619 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cccd76799a5140bc925f01e63abf45e2'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:46:59.911675 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-10de56b9-44f4-48cc-93c1-10b27c97e619 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cccd76799a5140bc925f01e63abf45e2'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:46:59.989056 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-10de56b9-44f4-48cc-93c1-10b27c97e619 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cccd76799a5140bc925f01e63abf45e2'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:00.000909 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486da08-435e-4544-8a69-2e601de2349e None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:00.001707 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6486da08-435e-4544-8a69-2e601de2349e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:00.010101 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1108/2228] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:46:59 2022] DELETE /identity/v3/users/cccd76799a5140bc925f01e63abf45e2 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:47:00.015705 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70372b-67c8-435e-b12e-f1845d1056d8 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:00.019810 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70372b-67c8-435e-b12e-f1845d1056d8 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:00.019810 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70372b-67c8-435e-b12e-f1845d1056d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:00.041126 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-db70372b-67c8-435e-b12e-f1845d1056d8 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:47:00.042382 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-db70372b-67c8-435e-b12e-f1845d1056d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:00.079951 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486da08-435e-4544-8a69-2e601de2349e None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:00.109196 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1109/2229] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:47:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:47:00.120424 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70372b-67c8-435e-b12e-f1845d1056d8 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:00.123738 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-db70372b-67c8-435e-b12e-f1845d1056d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:00.130096 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:47:00.153296 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a7e75-8a54-43c9-952f-4fbc900d4083 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e69c0e02ad944f2a85fc5318018e5cb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:00.153921 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a7e75-8a54-43c9-952f-4fbc900d4083 service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:00.154197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a7e75-8a54-43c9-952f-4fbc900d4083 service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:00.154463 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a7e75-8a54-43c9-952f-4fbc900d4083 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:00.156361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a67a7e75-8a54-43c9-952f-4fbc900d4083 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:00.187855 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70372b-67c8-435e-b12e-f1845d1056d8 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:00.191609 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23bb90e-e943-4fe2-92f0-83338a54d991 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:00.192215 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bb90e-e943-4fe2-92f0-83338a54d991 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:00.192490 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bb90e-e943-4fe2-92f0-83338a54d991 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:00.192756 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bb90e-e943-4fe2-92f0-83338a54d991 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:00.202361 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23bb90e-e943-4fe2-92f0-83338a54d991 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:00.203781 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23bb90e-e943-4fe2-92f0-83338a54d991 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:00.214075 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1110/2230] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:47:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:47:00.217583 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a7e75-8a54-43c9-952f-4fbc900d4083 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:00.219197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a7e75-8a54-43c9-952f-4fbc900d4083 service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:00.227739 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1121/2231] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 08:46:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 08:47:00.655388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23bb90e-e943-4fe2-92f0-83338a54d991 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:00.657351 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c23bb90e-e943-4fe2-92f0-83338a54d991 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:00.688645 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23bb90e-e943-4fe2-92f0-83338a54d991 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:00.703390 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606bb12-8f28-48e5-99db-c66ffc1fb716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:00.703390 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606bb12-8f28-48e5-99db-c66ffc1fb716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:00.703390 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606bb12-8f28-48e5-99db-c66ffc1fb716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:00.703390 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606bb12-8f28-48e5-99db-c66ffc1fb716 admin admin] PATH_INFO: `/v3/projects/a263438c78bf4c34be8f427c12d8a291` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:00.712011 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606bb12-8f28-48e5-99db-c66ffc1fb716 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a263438c78bf4c34be8f427c12d8a291)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:00.714033 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606bb12-8f28-48e5-99db-c66ffc1fb716 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:00.724562 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:47:00.885548 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a606bb12-8f28-48e5-99db-c66ffc1fb716 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a263438c78bf4c34be8f427c12d8a291', 'request_id': 'req-a606bb12-8f28-48e5-99db-c66ffc1fb716'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:00.892717 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a606bb12-8f28-48e5-99db-c66ffc1fb716 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a263438c78bf4c34be8f427c12d8a291', 'request_id': 'req-a606bb12-8f28-48e5-99db-c66ffc1fb716'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:00.899600 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a606bb12-8f28-48e5-99db-c66ffc1fb716 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a263438c78bf4c34be8f427c12d8a291', 'request_id': 'req-a606bb12-8f28-48e5-99db-c66ffc1fb716'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:00.995443 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a606bb12-8f28-48e5-99db-c66ffc1fb716 admin admin] The token cache is being invalidate because project a263438c78bf4c34be8f427c12d8a291 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:47:00.995443 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a606bb12-8f28-48e5-99db-c66ffc1fb716 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:00.995443 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-a606bb12-8f28-48e5-99db-c66ffc1fb716 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:01.001109 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1111/2232] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:47:00 2022] DELETE /identity/v3/projects/a263438c78bf4c34be8f427c12d8a291 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:47:01.003903 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8caac-4ee5-437d-9b5f-bd1a1872a84b None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:01.003903 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8caac-4ee5-437d-9b5f-bd1a1872a84b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:01.003903 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8caac-4ee5-437d-9b5f-bd1a1872a84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:01.032396 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-cdb8caac-4ee5-437d-9b5f-bd1a1872a84b None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:47:01.033732 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-cdb8caac-4ee5-437d-9b5f-bd1a1872a84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:01.113302 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1122/2233] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:47:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:47:01.117548 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236d01a-c498-41c4-94ac-0e059241cebb None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:01.117871 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236d01a-c498-41c4-94ac-0e059241cebb None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:01.118238 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236d01a-c498-41c4-94ac-0e059241cebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:01.141110 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-1236d01a-c498-41c4-94ac-0e059241cebb None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:47:01.142565 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1236d01a-c498-41c4-94ac-0e059241cebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:01.154244 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1112/2234] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:47:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:47:01.155506 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb8caac-4ee5-437d-9b5f-bd1a1872a84b None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:01.166510 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb8caac-4ee5-437d-9b5f-bd1a1872a84b None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:01.171854 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca1d73-2575-4666-8072-b3cf6cd594f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:01.172467 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca1d73-2575-4666-8072-b3cf6cd594f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:01.172764 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca1d73-2575-4666-8072-b3cf6cd594f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:01.173015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca1d73-2575-4666-8072-b3cf6cd594f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:01.173695 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca1d73-2575-4666-8072-b3cf6cd594f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:47:01.173995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca1d73-2575-4666-8072-b3cf6cd594f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:01.177372 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca1d73-2575-4666-8072-b3cf6cd594f4 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:01.188372 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1123/2235] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:47:01.195112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1236d01a-c498-41c4-94ac-0e059241cebb None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:01.203755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1236d01a-c498-41c4-94ac-0e059241cebb None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:01.210452 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519f315-1f98-4356-8d8e-7826b0da67ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:01.213476 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519f315-1f98-4356-8d8e-7826b0da67ee admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:01.213476 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519f315-1f98-4356-8d8e-7826b0da67ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:01.213476 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519f315-1f98-4356-8d8e-7826b0da67ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:01.213476 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519f315-1f98-4356-8d8e-7826b0da67ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:01.214503 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519f315-1f98-4356-8d8e-7826b0da67ee admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:01.312644 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1113/2236] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:47:01 2022] POST /identity/v3/projects => generated 357 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:47:01.317085 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca1d73-2575-4666-8072-b3cf6cd594f4 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:01.328492 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca1d73-2575-4666-8072-b3cf6cd594f4 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:01.391939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3311ce-66fc-4367-91b7-512e49c66d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:01.392550 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3311ce-66fc-4367-91b7-512e49c66d2a admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:01.392857 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3311ce-66fc-4367-91b7-512e49c66d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:01.393128 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3311ce-66fc-4367-91b7-512e49c66d2a admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:01.393789 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3311ce-66fc-4367-91b7-512e49c66d2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:01.395711 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3311ce-66fc-4367-91b7-512e49c66d2a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:01.492213 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1124/2237] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:47:01 2022] POST /identity/v3/users => generated 279 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:47:01.504078 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519f315-1f98-4356-8d8e-7826b0da67ee admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:01.516794 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519f315-1f98-4356-8d8e-7826b0da67ee admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:01.521848 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e788b1-218e-48b9-9313-fa8c01b1785d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:01.522738 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e788b1-218e-48b9-9313-fa8c01b1785d admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:01.523148 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e788b1-218e-48b9-9313-fa8c01b1785d admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:01.523421 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e788b1-218e-48b9-9313-fa8c01b1785d admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:01.524093 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e788b1-218e-48b9-9313-fa8c01b1785d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:01.525716 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e788b1-218e-48b9-9313-fa8c01b1785d admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:01.541195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1114/2238] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:47:01.544732 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3311ce-66fc-4367-91b7-512e49c66d2a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:01.555470 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3311ce-66fc-4367-91b7-512e49c66d2a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:01.560954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b920be-3f5b-4f11-b2a5-65a6e3be2212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:01.561609 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b920be-3f5b-4f11-b2a5-65a6e3be2212 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:01.562010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b920be-3f5b-4f11-b2a5-65a6e3be2212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:01.562349 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b920be-3f5b-4f11-b2a5-65a6e3be2212 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:01.563083 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b920be-3f5b-4f11-b2a5-65a6e3be2212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:01.564854 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b920be-3f5b-4f11-b2a5-65a6e3be2212 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:01.566636 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:47:01.578078 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1125/2239] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:47:01.583452 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e788b1-218e-48b9-9313-fa8c01b1785d admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:01.594361 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e788b1-218e-48b9-9313-fa8c01b1785d admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:01.600198 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163b8bf-5068-4193-9e7b-c605154e98be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:01.601247 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163b8bf-5068-4193-9e7b-c605154e98be admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:01.601599 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163b8bf-5068-4193-9e7b-c605154e98be admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:01.601940 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163b8bf-5068-4193-9e7b-c605154e98be admin admin] PATH_INFO: `/v3/projects/1d5e61d3e4f64cb08a8760df513bab75/users/1f9cc29fd6bc470ea50154a9a45ea9d1/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:01.622043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163b8bf-5068-4193-9e7b-c605154e98be admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d5e61d3e4f64cb08a8760df513bab75, user_id=1f9cc29fd6bc470ea50154a9a45ea9d1, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:01.625334 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163b8bf-5068-4193-9e7b-c605154e98be admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:01.701793 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1115/2240] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:47:01 2022] PUT /identity/v3/projects/1d5e61d3e4f64cb08a8760df513bab75/users/1f9cc29fd6bc470ea50154a9a45ea9d1/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:47:01.713047 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c9154-4ab7-46b0-bb1f-0bd933eba966 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:01.713361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c9154-4ab7-46b0-bb1f-0bd933eba966 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:01.713642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c9154-4ab7-46b0-bb1f-0bd933eba966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:01.755379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-412c9154-4ab7-46b0-bb1f-0bd933eba966 None None] MFA Rules not processed for user `1f9cc29fd6bc470ea50154a9a45ea9d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:47:01.757128 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-412c9154-4ab7-46b0-bb1f-0bd933eba966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:01.836386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1126/2241] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:47:01 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:47:01.841176 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163b8bf-5068-4193-9e7b-c605154e98be admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:01.922909 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163b8bf-5068-4193-9e7b-c605154e98be admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:01.927976 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d2f46-9a41-4e7a-bd35-5cf449ec7a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:01.928586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d2f46-9a41-4e7a-bd35-5cf449ec7a90 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:01.932112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d2f46-9a41-4e7a-bd35-5cf449ec7a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:01.932112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d2f46-9a41-4e7a-bd35-5cf449ec7a90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:01.932112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d2f46-9a41-4e7a-bd35-5cf449ec7a90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:01.932112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d2f46-9a41-4e7a-bd35-5cf449ec7a90 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:02.048782 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1116/2242] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:47:01 2022] POST /identity/v3/projects => generated 355 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:47:02.052930 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c9154-4ab7-46b0-bb1f-0bd933eba966 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:02.065352 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c9154-4ab7-46b0-bb1f-0bd933eba966 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:02.132422 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d766607-2322-41e1-b83b-1ae124b28dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:02.133218 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d766607-2322-41e1-b83b-1ae124b28dcf admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:02.133603 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d766607-2322-41e1-b83b-1ae124b28dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:02.133975 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d766607-2322-41e1-b83b-1ae124b28dcf admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:02.134710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d766607-2322-41e1-b83b-1ae124b28dcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:02.136787 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d766607-2322-41e1-b83b-1ae124b28dcf admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:02.221671 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1127/2243] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:47:02 2022] POST /identity/v3/users => generated 284 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:47:02.226367 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d2f46-9a41-4e7a-bd35-5cf449ec7a90 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:02.239054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d2f46-9a41-4e7a-bd35-5cf449ec7a90 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:02.244972 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874dde9-f53b-4050-9c89-d5647c9f125f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:02.245806 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874dde9-f53b-4050-9c89-d5647c9f125f admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:02.246325 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874dde9-f53b-4050-9c89-d5647c9f125f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:02.246696 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874dde9-f53b-4050-9c89-d5647c9f125f admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:02.247600 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874dde9-f53b-4050-9c89-d5647c9f125f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:02.249333 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874dde9-f53b-4050-9c89-d5647c9f125f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:02.265303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1117/2244] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:47:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:47:02.266855 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d766607-2322-41e1-b83b-1ae124b28dcf admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:02.278206 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d766607-2322-41e1-b83b-1ae124b28dcf admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:02.283041 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dec89f-964d-455a-92ff-7ec06bfd0962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:02.283808 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dec89f-964d-455a-92ff-7ec06bfd0962 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:02.284192 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dec89f-964d-455a-92ff-7ec06bfd0962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:02.284570 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dec89f-964d-455a-92ff-7ec06bfd0962 admin admin] PATH_INFO: `/v3/projects/c753b80cd0e942d4841dcb0ea2d1b412/users/1770ef1037d64ec4aa1e02894edad521/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:02.301653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dec89f-964d-455a-92ff-7ec06bfd0962 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c753b80cd0e942d4841dcb0ea2d1b412, user_id=1770ef1037d64ec4aa1e02894edad521, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:02.304111 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dec89f-964d-455a-92ff-7ec06bfd0962 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:02.386288 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1128/2245] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:47:02 2022] PUT /identity/v3/projects/c753b80cd0e942d4841dcb0ea2d1b412/users/1770ef1037d64ec4aa1e02894edad521/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:47:02.387707 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874dde9-f53b-4050-9c89-d5647c9f125f admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:02.394680 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:47:02.405293 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874dde9-f53b-4050-9c89-d5647c9f125f admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:02.480589 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207ad560-2a3d-4b7c-a744-6409938d7fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:02.481297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ad560-2a3d-4b7c-a744-6409938d7fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:02.481573 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ad560-2a3d-4b7c-a744-6409938d7fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:02.481850 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ad560-2a3d-4b7c-a744-6409938d7fc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:02.482505 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207ad560-2a3d-4b7c-a744-6409938d7fc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:02.484350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207ad560-2a3d-4b7c-a744-6409938d7fc0 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:02.499341 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1118/2246] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:47:02 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:47:02.503939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dec89f-964d-455a-92ff-7ec06bfd0962 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:02.517240 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dec89f-964d-455a-92ff-7ec06bfd0962 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:02.522043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5191c-2d98-48ef-92a2-5bef1bc544cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:02.522773 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5191c-2d98-48ef-92a2-5bef1bc544cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:02.523125 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5191c-2d98-48ef-92a2-5bef1bc544cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:02.523472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5191c-2d98-48ef-92a2-5bef1bc544cf admin admin] PATH_INFO: `/v3/projects/c753b80cd0e942d4841dcb0ea2d1b412/users/1770ef1037d64ec4aa1e02894edad521/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:02.525807 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5191c-2d98-48ef-92a2-5bef1bc544cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c753b80cd0e942d4841dcb0ea2d1b412, user_id=1770ef1037d64ec4aa1e02894edad521, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:02.528574 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5191c-2d98-48ef-92a2-5bef1bc544cf admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:02.540663 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1129/2247] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:47:02 2022] PUT /identity/v3/projects/c753b80cd0e942d4841dcb0ea2d1b412/users/1770ef1037d64ec4aa1e02894edad521/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:47:02.552677 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37447cf-ed27-4f9d-966c-6017b62ac148 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:02.552973 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37447cf-ed27-4f9d-966c-6017b62ac148 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:02.553275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37447cf-ed27-4f9d-966c-6017b62ac148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:02.584134 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-e37447cf-ed27-4f9d-966c-6017b62ac148 None None] MFA Rules not processed for user `1770ef1037d64ec4aa1e02894edad521`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:47:02.585381 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e37447cf-ed27-4f9d-966c-6017b62ac148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:02.663275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1119/2248] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:47:02 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:47:02.675205 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5191c-2d98-48ef-92a2-5bef1bc544cf admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:02.677049 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9bb5191c-2d98-48ef-92a2-5bef1bc544cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:02.729774 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:47:02.747725 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5191c-2d98-48ef-92a2-5bef1bc544cf admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:02.751407 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f3585-5435-4099-b08d-4d19378c7a1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:02.752020 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f3585-5435-4099-b08d-4d19378c7a1f service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:02.752296 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f3585-5435-4099-b08d-4d19378c7a1f service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:02.752565 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f3585-5435-4099-b08d-4d19378c7a1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:02.766982 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f3585-5435-4099-b08d-4d19378c7a1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:02.768542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f3585-5435-4099-b08d-4d19378c7a1f service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:02.780719 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1130/2249] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:47:02 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:47:03.215194 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37447cf-ed27-4f9d-966c-6017b62ac148 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:03.226330 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37447cf-ed27-4f9d-966c-6017b62ac148 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:03.230018 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af732b-c676-4dc3-91ab-9522f6eaa6bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:03.231207 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af732b-c676-4dc3-91ab-9522f6eaa6bf service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:03.231481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af732b-c676-4dc3-91ab-9522f6eaa6bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:03.231752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af732b-c676-4dc3-91ab-9522f6eaa6bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:03.325988 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af732b-c676-4dc3-91ab-9522f6eaa6bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:03.327831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af732b-c676-4dc3-91ab-9522f6eaa6bf service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:03.342916 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1120/2250] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:47:03 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:47:10.731870 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f3585-5435-4099-b08d-4d19378c7a1f service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:10.733472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ab8f3585-5435-4099-b08d-4d19378c7a1f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:10.835917 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f3585-5435-4099-b08d-4d19378c7a1f service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:10.842838 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799389d2-ddd6-4e7b-9f02-90fba4290c3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:10.843063 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799389d2-ddd6-4e7b-9f02-90fba4290c3a service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:10.843210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799389d2-ddd6-4e7b-9f02-90fba4290c3a service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:10.843458 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799389d2-ddd6-4e7b-9f02-90fba4290c3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:10.845281 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-799389d2-ddd6-4e7b-9f02-90fba4290c3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:10.877386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799389d2-ddd6-4e7b-9f02-90fba4290c3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:10.877386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799389d2-ddd6-4e7b-9f02-90fba4290c3a service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:10.891130 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1131/2251] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:47:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:47:11.405585 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af732b-c676-4dc3-91ab-9522f6eaa6bf service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:11.407173 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-84af732b-c676-4dc3-91ab-9522f6eaa6bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:11.445246 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:47:11.479962 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af732b-c676-4dc3-91ab-9522f6eaa6bf service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:11.483655 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2983548-f490-4470-ac82-9c80714f8749 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:11.489282 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2983548-f490-4470-ac82-9c80714f8749 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:11.489282 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2983548-f490-4470-ac82-9c80714f8749 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:11.489282 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2983548-f490-4470-ac82-9c80714f8749 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:11.489282 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c2983548-f490-4470-ac82-9c80714f8749 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:11.499197 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2983548-f490-4470-ac82-9c80714f8749 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:11.500488 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2983548-f490-4470-ac82-9c80714f8749 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:11.510303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1121/2252] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:47:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:47:12.487055 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56629db-d597-4633-8cb5-cd541c14f653 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:12.487547 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56629db-d597-4633-8cb5-cd541c14f653 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:12.487644 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56629db-d597-4633-8cb5-cd541c14f653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:12.508112 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a56629db-d597-4633-8cb5-cd541c14f653 None None] MFA Rules not processed for user `cd870619247a47ac810fea1aa636ac65`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:47:12.510120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a56629db-d597-4633-8cb5-cd541c14f653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:12.569559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1132/2253] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:47:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:47:12.897772 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2983548-f490-4470-ac82-9c80714f8749 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:12.907120 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2983548-f490-4470-ac82-9c80714f8749 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:12.910757 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d153e3-6044-4c5d-873d-f12c6cf52656 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:12.911353 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d153e3-6044-4c5d-873d-f12c6cf52656 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:12.911629 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d153e3-6044-4c5d-873d-f12c6cf52656 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:12.911897 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d153e3-6044-4c5d-873d-f12c6cf52656 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:12.913719 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-96d153e3-6044-4c5d-873d-f12c6cf52656 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:12.924181 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d153e3-6044-4c5d-873d-f12c6cf52656 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:12.925623 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d153e3-6044-4c5d-873d-f12c6cf52656 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:12.933784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1122/2254] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:47:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:47:13.726511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56629db-d597-4633-8cb5-cd541c14f653 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:13.728818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a56629db-d597-4633-8cb5-cd541c14f653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:13.798102 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56629db-d597-4633-8cb5-cd541c14f653 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:13.803014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e32480-a845-48c5-9b69-ce454aa7741e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c23a782b9b4144bace1248c3b2e754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:13.803620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e32480-a845-48c5-9b69-ce454aa7741e service placement] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:13.803900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e32480-a845-48c5-9b69-ce454aa7741e service placement] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:13.804173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e32480-a845-48c5-9b69-ce454aa7741e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:13.805885 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a2e32480-a845-48c5-9b69-ce454aa7741e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:13.817168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e32480-a845-48c5-9b69-ce454aa7741e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:13.818781 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e32480-a845-48c5-9b69-ce454aa7741e service placement] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:13.824119 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:47:13.829413 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1133/2255] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:47:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:47:17.821399 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa70ad2-8e97-4d00-a1ad-0c6c7cd2c72e None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:17.821913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa70ad2-8e97-4d00-a1ad-0c6c7cd2c72e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:17.821999 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa70ad2-8e97-4d00-a1ad-0c6c7cd2c72e None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:17.823347 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1123/2256] 10.222.0.33 () {54 vars in 921 bytes} [Mon Aug 8 08:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:47:17.838916 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8c2e1-1945-4947-ad32-a1d82c753fe6 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:17.838916 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8c2e1-1945-4947-ad32-a1d82c753fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:17.838916 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8c2e1-1945-4947-ad32-a1d82c753fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:17.859614 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-60f8c2e1-1945-4947-ad32-a1d82c753fe6 None None] MFA Rules not processed for user `a3317f751ec74bde95ec262394001292`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:47:17.860818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-60f8c2e1-1945-4947-ad32-a1d82c753fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:17.870386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1134/2257] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Aug 8 08:47:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:47:17.883245 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa70ad2-8e97-4d00-a1ad-0c6c7cd2c72e None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:17.885213 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6aa70ad2-8e97-4d00-a1ad-0c6c7cd2c72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:17.948790 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa70ad2-8e97-4d00-a1ad-0c6c7cd2c72e None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:17.953357 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b14d59-0928-42b5-9698-f85be7bbfe40 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '627171bf2a2e4a21ba6f800879f8c665', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:17.954155 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b14d59-0928-42b5-9698-f85be7bbfe40 service glance] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:17.954429 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b14d59-0928-42b5-9698-f85be7bbfe40 service glance] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:17.954995 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b14d59-0928-42b5-9698-f85be7bbfe40 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:17.968689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b14d59-0928-42b5-9698-f85be7bbfe40 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:17.971127 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b14d59-0928-42b5-9698-f85be7bbfe40 service glance] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:17.992924 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1124/2258] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:47:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:47:18.207157 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69991ae1-b2c8-4fcd-85bc-6691ee014748 None None] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:18.207449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69991ae1-b2c8-4fcd-85bc-6691ee014748 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:18.207903 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69991ae1-b2c8-4fcd-85bc-6691ee014748 None None] PATH_INFO: `/` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:18.209214 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1135/2259] 10.222.0.33 () {54 vars in 921 bytes} [Mon Aug 8 08:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:47:18.213818 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6334512-e4c5-4901-8659-de47f0300aa0 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:18.214298 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6334512-e4c5-4901-8659-de47f0300aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:18.214595 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6334512-e4c5-4901-8659-de47f0300aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:18.234477 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-d6334512-e4c5-4901-8659-de47f0300aa0 None None] MFA Rules not processed for user `2c3b8b43026c40209bfc42acd2830610`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:47:18.235822 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d6334512-e4c5-4901-8659-de47f0300aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:18.242543 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1125/2260] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Aug 8 08:47:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:47:18.254979 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69991ae1-b2c8-4fcd-85bc-6691ee014748 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:18.257485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-69991ae1-b2c8-4fcd-85bc-6691ee014748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:18.274225 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69991ae1-b2c8-4fcd-85bc-6691ee014748 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:18.278917 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df93f7-2961-46cc-aa88-b24499ca9276 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:18.279721 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df93f7-2961-46cc-aa88-b24499ca9276 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:18.280001 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df93f7-2961-46cc-aa88-b24499ca9276 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:18.280272 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df93f7-2961-46cc-aa88-b24499ca9276 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:18.287769 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df93f7-2961-46cc-aa88-b24499ca9276 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:18.289252 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df93f7-2961-46cc-aa88-b24499ca9276 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:18.297970 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1136/2261] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:47:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:47:18.387958 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d9222-6597-4966-9e5d-f6e22351896f None None] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:18.388559 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d9222-6597-4966-9e5d-f6e22351896f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:18.388908 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d9222-6597-4966-9e5d-f6e22351896f None None] PATH_INFO: `/` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:18.390597 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1126/2262] 10.222.0.33 () {54 vars in 921 bytes} [Mon Aug 8 08:47:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 08:47:18.396210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3741c86-4e45-40ad-8bcb-890404b2900c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:18.396489 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3741c86-4e45-40ad-8bcb-890404b2900c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:18.396755 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3741c86-4e45-40ad-8bcb-890404b2900c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:18.416524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b3741c86-4e45-40ad-8bcb-890404b2900c None None] MFA Rules not processed for user `a3317f751ec74bde95ec262394001292`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:47:18.417745 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b3741c86-4e45-40ad-8bcb-890404b2900c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:18.427863 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1137/2263] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Aug 8 08:47:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:47:18.438924 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d9222-6597-4966-9e5d-f6e22351896f None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:18.450276 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d9222-6597-4966-9e5d-f6e22351896f None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:18.453996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4dcbf-67e1-4d15-8b13-545fd38877b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:18.454590 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4dcbf-67e1-4d15-8b13-545fd38877b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:18.454862 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4dcbf-67e1-4d15-8b13-545fd38877b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:18.455127 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4dcbf-67e1-4d15-8b13-545fd38877b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:18.465369 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e4dcbf-67e1-4d15-8b13-545fd38877b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:18.466797 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e4dcbf-67e1-4d15-8b13-545fd38877b8 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:18.478956 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1127/2264] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:47:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:47:20.805163 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b20d19-c9a7-4d6d-87fe-2774e649d1e3 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:20.805631 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b20d19-c9a7-4d6d-87fe-2774e649d1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:20.805837 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b20d19-c9a7-4d6d-87fe-2774e649d1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:20.826274 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-20b20d19-c9a7-4d6d-87fe-2774e649d1e3 None None] MFA Rules not processed for user `7f3422c6f9a041ceba851b71bd15d0df`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:47:20.827490 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-20b20d19-c9a7-4d6d-87fe-2774e649d1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:20.883033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1138/2265] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 08:47:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:47:20.895743 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4dcbf-67e1-4d15-8b13-545fd38877b8 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:20.897116 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f3e4dcbf-67e1-4d15-8b13-545fd38877b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:20.965087 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4dcbf-67e1-4d15-8b13-545fd38877b8 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:20.970624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5cdf7-1331-4c52-9f8a-85db65a4fbbd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e69c0e02ad944f2a85fc5318018e5cb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:20.971296 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5cdf7-1331-4c52-9f8a-85db65a4fbbd service swift] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:20.971880 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5cdf7-1331-4c52-9f8a-85db65a4fbbd service swift] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:20.972032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5cdf7-1331-4c52-9f8a-85db65a4fbbd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:20.980500 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5cdf7-1331-4c52-9f8a-85db65a4fbbd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:20.982136 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5cdf7-1331-4c52-9f8a-85db65a4fbbd service swift] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:20.989973 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1128/2266] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 08:47:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 08:47:22.203416 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b20d19-c9a7-4d6d-87fe-2774e649d1e3 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:22.205095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-20b20d19-c9a7-4d6d-87fe-2774e649d1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:22.221420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b20d19-c9a7-4d6d-87fe-2774e649d1e3 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:22.227131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0488167-8dfd-4f50-80fd-a237d837f468 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:22.227992 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0488167-8dfd-4f50-80fd-a237d837f468 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:22.228347 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0488167-8dfd-4f50-80fd-a237d837f468 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:22.229020 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0488167-8dfd-4f50-80fd-a237d837f468 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:22.231655 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e0488167-8dfd-4f50-80fd-a237d837f468 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:22.299806 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0488167-8dfd-4f50-80fd-a237d837f468 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:22.301448 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0488167-8dfd-4f50-80fd-a237d837f468 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:22.312405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1139/2267] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:47:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:47:23.402166 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5cdf7-1331-4c52-9f8a-85db65a4fbbd service swift] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:23.439954 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:47:23.474929 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5cdf7-1331-4c52-9f8a-85db65a4fbbd service swift] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:23.479985 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b2038e-108e-4675-8888-31cf52271361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:23.480715 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2038e-108e-4675-8888-31cf52271361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:23.481200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2038e-108e-4675-8888-31cf52271361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:23.481551 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2038e-108e-4675-8888-31cf52271361 admin admin] PATH_INFO: `/v3/users/1f9cc29fd6bc470ea50154a9a45ea9d1` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:23.482778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b2038e-108e-4675-8888-31cf52271361 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f9cc29fd6bc470ea50154a9a45ea9d1)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:23.485215 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b2038e-108e-4675-8888-31cf52271361 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:23.713469 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-15b2038e-108e-4675-8888-31cf52271361 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f9cc29fd6bc470ea50154a9a45ea9d1'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:23.714123 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-15b2038e-108e-4675-8888-31cf52271361 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f9cc29fd6bc470ea50154a9a45ea9d1'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:23.714999 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-15b2038e-108e-4675-8888-31cf52271361 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f9cc29fd6bc470ea50154a9a45ea9d1'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:23.722291 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-15b2038e-108e-4675-8888-31cf52271361 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f9cc29fd6bc470ea50154a9a45ea9d1'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:23.787632 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-15b2038e-108e-4675-8888-31cf52271361 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f9cc29fd6bc470ea50154a9a45ea9d1'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:23.806835 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1129/2268] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:47:23 2022] DELETE /identity/v3/users/1f9cc29fd6bc470ea50154a9a45ea9d1 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:47:23.812095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbe3cd-d0d6-4ac0-a271-fd31eb4547b6 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:23.812613 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbe3cd-d0d6-4ac0-a271-fd31eb4547b6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:23.812711 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbe3cd-d0d6-4ac0-a271-fd31eb4547b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:23.842486 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-31fbe3cd-d0d6-4ac0-a271-fd31eb4547b6 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:47:23.843984 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-31fbe3cd-d0d6-4ac0-a271-fd31eb4547b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:23.919365 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1140/2269] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:47:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:47:23.935233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b2038e-108e-4675-8888-31cf52271361 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:23.935654 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-15b2038e-108e-4675-8888-31cf52271361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:24.011252 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b2038e-108e-4675-8888-31cf52271361 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:24.029108 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e69541-f35a-4189-951f-ba9a224f3e3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:24.029910 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e69541-f35a-4189-951f-ba9a224f3e3f service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:24.030296 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e69541-f35a-4189-951f-ba9a224f3e3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:24.031086 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e69541-f35a-4189-951f-ba9a224f3e3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:24.047020 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e69541-f35a-4189-951f-ba9a224f3e3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:24.049043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e69541-f35a-4189-951f-ba9a224f3e3f service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:24.074393 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1130/2270] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:47:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:47:24.659289 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbe3cd-d0d6-4ac0-a271-fd31eb4547b6 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:24.661563 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-31fbe3cd-d0d6-4ac0-a271-fd31eb4547b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:24.678360 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbe3cd-d0d6-4ac0-a271-fd31eb4547b6 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:24.684203 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892fca6a-ab29-4e2a-927c-34eb6b2caae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:24.684858 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fca6a-ab29-4e2a-927c-34eb6b2caae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:24.685130 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fca6a-ab29-4e2a-927c-34eb6b2caae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:24.685430 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fca6a-ab29-4e2a-927c-34eb6b2caae1 admin admin] PATH_INFO: `/v3/projects/1d5e61d3e4f64cb08a8760df513bab75` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:24.687401 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892fca6a-ab29-4e2a-927c-34eb6b2caae1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d5e61d3e4f64cb08a8760df513bab75)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:24.689708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892fca6a-ab29-4e2a-927c-34eb6b2caae1 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:24.703682 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:47:24.818400 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-892fca6a-ab29-4e2a-927c-34eb6b2caae1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d5e61d3e4f64cb08a8760df513bab75', 'request_id': 'req-892fca6a-ab29-4e2a-927c-34eb6b2caae1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:24.884958 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-892fca6a-ab29-4e2a-927c-34eb6b2caae1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d5e61d3e4f64cb08a8760df513bab75', 'request_id': 'req-892fca6a-ab29-4e2a-927c-34eb6b2caae1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:24.892709 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-892fca6a-ab29-4e2a-927c-34eb6b2caae1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d5e61d3e4f64cb08a8760df513bab75', 'request_id': 'req-892fca6a-ab29-4e2a-927c-34eb6b2caae1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:24.905479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-892fca6a-ab29-4e2a-927c-34eb6b2caae1 admin admin] The token cache is being invalidate because project 1d5e61d3e4f64cb08a8760df513bab75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:47:24.905776 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-892fca6a-ab29-4e2a-927c-34eb6b2caae1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:24.906451 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-892fca6a-ab29-4e2a-927c-34eb6b2caae1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:24.912165 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e69541-f35a-4189-951f-ba9a224f3e3f service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:24.912805 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1141/2271] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:47:24 2022] DELETE /identity/v3/projects/1d5e61d3e4f64cb08a8760df513bab75 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:47:24.923415 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e69541-f35a-4189-951f-ba9a224f3e3f service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:24.971830 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:47:24.984877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9780b1f-a29d-4444-a99f-b7e752ddd0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:24.985633 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9780b1f-a29d-4444-a99f-b7e752ddd0c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:24.986027 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9780b1f-a29d-4444-a99f-b7e752ddd0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:24.986383 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9780b1f-a29d-4444-a99f-b7e752ddd0c5 admin admin] PATH_INFO: `/v3/users/1770ef1037d64ec4aa1e02894edad521` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:24.987584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9780b1f-a29d-4444-a99f-b7e752ddd0c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1770ef1037d64ec4aa1e02894edad521)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:24.989637 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9780b1f-a29d-4444-a99f-b7e752ddd0c5 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:25.227096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e9780b1f-a29d-4444-a99f-b7e752ddd0c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1770ef1037d64ec4aa1e02894edad521'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:25.227745 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e9780b1f-a29d-4444-a99f-b7e752ddd0c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1770ef1037d64ec4aa1e02894edad521'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:25.228435 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e9780b1f-a29d-4444-a99f-b7e752ddd0c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1770ef1037d64ec4aa1e02894edad521'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:25.240638 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e9780b1f-a29d-4444-a99f-b7e752ddd0c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1770ef1037d64ec4aa1e02894edad521'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:25.318898 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e9780b1f-a29d-4444-a99f-b7e752ddd0c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1770ef1037d64ec4aa1e02894edad521'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:25.343953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1131/2272] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:47:24 2022] DELETE /identity/v3/users/1770ef1037d64ec4aa1e02894edad521 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:47:25.910412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892fca6a-ab29-4e2a-927c-34eb6b2caae1 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:25.918702 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-892fca6a-ab29-4e2a-927c-34eb6b2caae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:25.927663 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892fca6a-ab29-4e2a-927c-34eb6b2caae1 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:26.003784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e295ba-c809-4809-82ac-ed0d721ba5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:26.004548 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e295ba-c809-4809-82ac-ed0d721ba5eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:26.005120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e295ba-c809-4809-82ac-ed0d721ba5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:26.005424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e295ba-c809-4809-82ac-ed0d721ba5eb admin admin] PATH_INFO: `/v3/projects/c753b80cd0e942d4841dcb0ea2d1b412` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:26.006545 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e295ba-c809-4809-82ac-ed0d721ba5eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c753b80cd0e942d4841dcb0ea2d1b412)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:26.008528 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e295ba-c809-4809-82ac-ed0d721ba5eb admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:26.122223 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c9e295ba-c809-4809-82ac-ed0d721ba5eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c753b80cd0e942d4841dcb0ea2d1b412', 'request_id': 'req-c9e295ba-c809-4809-82ac-ed0d721ba5eb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:26.193086 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c9e295ba-c809-4809-82ac-ed0d721ba5eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c753b80cd0e942d4841dcb0ea2d1b412', 'request_id': 'req-c9e295ba-c809-4809-82ac-ed0d721ba5eb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:26.200982 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c9e295ba-c809-4809-82ac-ed0d721ba5eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c753b80cd0e942d4841dcb0ea2d1b412', 'request_id': 'req-c9e295ba-c809-4809-82ac-ed0d721ba5eb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:26.212747 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c9e295ba-c809-4809-82ac-ed0d721ba5eb admin admin] The token cache is being invalidate because project c753b80cd0e942d4841dcb0ea2d1b412 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:47:26.213062 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c9e295ba-c809-4809-82ac-ed0d721ba5eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:26.213749 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c9e295ba-c809-4809-82ac-ed0d721ba5eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:26.226062 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1142/2273] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:47:25 2022] DELETE /identity/v3/projects/c753b80cd0e942d4841dcb0ea2d1b412 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:47:26.227358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771044b2-b140-4827-a46f-b38850098d42 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:26.227918 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771044b2-b140-4827-a46f-b38850098d42 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:26.228082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771044b2-b140-4827-a46f-b38850098d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:26.250034 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-771044b2-b140-4827-a46f-b38850098d42 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:47:26.251284 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-771044b2-b140-4827-a46f-b38850098d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:26.321518 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1132/2274] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:47:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:47:26.330060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb5d1f-5d24-4074-9e56-f77455c8a73b None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:26.330348 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb5d1f-5d24-4074-9e56-f77455c8a73b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:26.330617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb5d1f-5d24-4074-9e56-f77455c8a73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:26.364165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a8eb5d1f-5d24-4074-9e56-f77455c8a73b None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:47:26.365523 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a8eb5d1f-5d24-4074-9e56-f77455c8a73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:26.375026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1143/2275] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:47:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:47:26.378610 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771044b2-b140-4827-a46f-b38850098d42 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:26.396701 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771044b2-b140-4827-a46f-b38850098d42 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:26.403269 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd849dc-1804-411b-9203-bb0ad16a4115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:26.404023 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd849dc-1804-411b-9203-bb0ad16a4115 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:26.404419 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd849dc-1804-411b-9203-bb0ad16a4115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:26.404882 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd849dc-1804-411b-9203-bb0ad16a4115 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:26.405724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd849dc-1804-411b-9203-bb0ad16a4115 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:47:26.406166 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd849dc-1804-411b-9203-bb0ad16a4115 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:26.411873 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd849dc-1804-411b-9203-bb0ad16a4115 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:26.432370 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1133/2276] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:47:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:47:26.437495 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eb5d1f-5d24-4074-9e56-f77455c8a73b None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:26.450236 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eb5d1f-5d24-4074-9e56-f77455c8a73b None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:26.455046 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e9d20a-8371-4a65-9afb-df1192057b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:26.455733 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9d20a-8371-4a65-9afb-df1192057b59 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:26.456007 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9d20a-8371-4a65-9afb-df1192057b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:26.456279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9d20a-8371-4a65-9afb-df1192057b59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:26.456894 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e9d20a-8371-4a65-9afb-df1192057b59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:26.459503 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e9d20a-8371-4a65-9afb-df1192057b59 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:26.578969 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1144/2277] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:47:26 2022] POST /identity/v3/projects => generated 365 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:47:26.583356 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd849dc-1804-411b-9203-bb0ad16a4115 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:26.594174 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd849dc-1804-411b-9203-bb0ad16a4115 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:26.658176 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:47:26.665504 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8083dc3-0672-4c93-a8cd-9757ad23300e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:26.666121 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8083dc3-0672-4c93-a8cd-9757ad23300e admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:26.666403 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8083dc3-0672-4c93-a8cd-9757ad23300e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:26.666675 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8083dc3-0672-4c93-a8cd-9757ad23300e admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:26.667312 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8083dc3-0672-4c93-a8cd-9757ad23300e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:26.670337 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8083dc3-0672-4c93-a8cd-9757ad23300e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:26.762822 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1134/2278] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:47:26 2022] POST /identity/v3/users => generated 283 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:47:26.764144 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e9d20a-8371-4a65-9afb-df1192057b59 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:26.770839 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:47:26.783436 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e9d20a-8371-4a65-9afb-df1192057b59 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:26.788646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e3e03-c3b1-4d97-9ff1-d54948cbc3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:26.789336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e3e03-c3b1-4d97-9ff1-d54948cbc3f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:26.789599 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e3e03-c3b1-4d97-9ff1-d54948cbc3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:26.789869 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e3e03-c3b1-4d97-9ff1-d54948cbc3f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:26.790528 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282e3e03-c3b1-4d97-9ff1-d54948cbc3f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:26.792159 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282e3e03-c3b1-4d97-9ff1-d54948cbc3f2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:26.806755 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1145/2279] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:47:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:47:26.808002 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8083dc3-0672-4c93-a8cd-9757ad23300e admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:26.829966 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8083dc3-0672-4c93-a8cd-9757ad23300e admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:26.835797 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73191fb9-4b27-4af3-adc6-5eb6f9ef63c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:26.836430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73191fb9-4b27-4af3-adc6-5eb6f9ef63c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:26.836706 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73191fb9-4b27-4af3-adc6-5eb6f9ef63c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:26.837129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73191fb9-4b27-4af3-adc6-5eb6f9ef63c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:26.838996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73191fb9-4b27-4af3-adc6-5eb6f9ef63c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:26.843586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73191fb9-4b27-4af3-adc6-5eb6f9ef63c6 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:26.852018 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1135/2280] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:47:26 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:47:26.856927 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e3e03-c3b1-4d97-9ff1-d54948cbc3f2 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:26.868912 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e3e03-c3b1-4d97-9ff1-d54948cbc3f2 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:26.881842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f57bd4d-a768-44af-aa3f-bc4c8dc15386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:26.885089 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f57bd4d-a768-44af-aa3f-bc4c8dc15386 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:26.885089 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f57bd4d-a768-44af-aa3f-bc4c8dc15386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:26.885089 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f57bd4d-a768-44af-aa3f-bc4c8dc15386 admin admin] PATH_INFO: `/v3/projects/9e0ed574f00e475c9af0e464895c4845/users/1a95dc87e62c4ac2a74bf7e82a8fae79/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:26.905703 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f57bd4d-a768-44af-aa3f-bc4c8dc15386 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e0ed574f00e475c9af0e464895c4845, user_id=1a95dc87e62c4ac2a74bf7e82a8fae79, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:26.912842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f57bd4d-a768-44af-aa3f-bc4c8dc15386 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:26.989881 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1146/2281] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:47:26 2022] PUT /identity/v3/projects/9e0ed574f00e475c9af0e464895c4845/users/1a95dc87e62c4ac2a74bf7e82a8fae79/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:47:27.002306 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22835bbc-3f56-42c5-98cb-3f507d60bbb3 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:27.002616 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22835bbc-3f56-42c5-98cb-3f507d60bbb3 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:27.002866 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22835bbc-3f56-42c5-98cb-3f507d60bbb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:27.045930 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-22835bbc-3f56-42c5-98cb-3f507d60bbb3 None None] MFA Rules not processed for user `1a95dc87e62c4ac2a74bf7e82a8fae79`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:47:27.047299 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-22835bbc-3f56-42c5-98cb-3f507d60bbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:27.116699 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1136/2282] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:47:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:47:27.121353 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f57bd4d-a768-44af-aa3f-bc4c8dc15386 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:27.192163 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f57bd4d-a768-44af-aa3f-bc4c8dc15386 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:27.200812 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff01325-063d-4a0c-8f58-5e2ddc7fbfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:27.201610 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff01325-063d-4a0c-8f58-5e2ddc7fbfe9 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:27.201880 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff01325-063d-4a0c-8f58-5e2ddc7fbfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:27.202224 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff01325-063d-4a0c-8f58-5e2ddc7fbfe9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:27.202923 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff01325-063d-4a0c-8f58-5e2ddc7fbfe9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:27.205015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff01325-063d-4a0c-8f58-5e2ddc7fbfe9 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:27.303679 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1147/2283] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:47:27 2022] POST /identity/v3/projects => generated 363 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:47:27.309194 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22835bbc-3f56-42c5-98cb-3f507d60bbb3 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:27.320750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22835bbc-3f56-42c5-98cb-3f507d60bbb3 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:27.381131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd768d3e-4663-40d9-bfe5-ce2ea40d5766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:27.381771 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd768d3e-4663-40d9-bfe5-ce2ea40d5766 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:27.382050 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd768d3e-4663-40d9-bfe5-ce2ea40d5766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:27.382334 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd768d3e-4663-40d9-bfe5-ce2ea40d5766 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:27.383064 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd768d3e-4663-40d9-bfe5-ce2ea40d5766 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:27.385009 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd768d3e-4663-40d9-bfe5-ce2ea40d5766 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:27.462317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1137/2284] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:47:27 2022] POST /identity/v3/users => generated 288 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:47:27.466534 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff01325-063d-4a0c-8f58-5e2ddc7fbfe9 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:27.477620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff01325-063d-4a0c-8f58-5e2ddc7fbfe9 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:27.484016 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07effa63-5036-40bd-9444-838ec8623af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:27.484937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07effa63-5036-40bd-9444-838ec8623af9 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:27.485519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07effa63-5036-40bd-9444-838ec8623af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:27.485519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07effa63-5036-40bd-9444-838ec8623af9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:27.486830 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07effa63-5036-40bd-9444-838ec8623af9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:27.488455 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07effa63-5036-40bd-9444-838ec8623af9 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:27.500677 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1148/2285] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:47:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:47:27.504951 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd768d3e-4663-40d9-bfe5-ce2ea40d5766 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:27.518105 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd768d3e-4663-40d9-bfe5-ce2ea40d5766 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:27.522758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a1233-99a6-479c-beb0-fcb5d5fae2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:27.523422 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a1233-99a6-479c-beb0-fcb5d5fae2d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:27.523699 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a1233-99a6-479c-beb0-fcb5d5fae2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:27.523971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a1233-99a6-479c-beb0-fcb5d5fae2d2 admin admin] PATH_INFO: `/v3/projects/f70bd60330a84833a2573fc32cde778b/users/a8aca9f149a5468f9207c5f42f724f05/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:27.542995 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a1233-99a6-479c-beb0-fcb5d5fae2d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f70bd60330a84833a2573fc32cde778b, user_id=a8aca9f149a5468f9207c5f42f724f05, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:27.545815 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a1233-99a6-479c-beb0-fcb5d5fae2d2 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:27.640530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1138/2286] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:47:27 2022] PUT /identity/v3/projects/f70bd60330a84833a2573fc32cde778b/users/a8aca9f149a5468f9207c5f42f724f05/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:47:27.645529 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07effa63-5036-40bd-9444-838ec8623af9 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:27.658943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07effa63-5036-40bd-9444-838ec8623af9 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:27.776458 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bfba70-74de-463c-8e90-1776545e05bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:27.777997 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfba70-74de-463c-8e90-1776545e05bd admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:27.778634 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfba70-74de-463c-8e90-1776545e05bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:27.778986 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfba70-74de-463c-8e90-1776545e05bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:27.780096 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bfba70-74de-463c-8e90-1776545e05bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:27.782239 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bfba70-74de-463c-8e90-1776545e05bd admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:27.796704 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1149/2287] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:47:27 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:47:27.801008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a1233-99a6-479c-beb0-fcb5d5fae2d2 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:27.816896 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a1233-99a6-479c-beb0-fcb5d5fae2d2 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:27.826543 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c3577-a10e-4cbc-a4df-19d6c5f22569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:27.827269 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c3577-a10e-4cbc-a4df-19d6c5f22569 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:27.827562 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c3577-a10e-4cbc-a4df-19d6c5f22569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:27.827831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c3577-a10e-4cbc-a4df-19d6c5f22569 admin admin] PATH_INFO: `/v3/projects/f70bd60330a84833a2573fc32cde778b/users/a8aca9f149a5468f9207c5f42f724f05/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:27.830129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c3577-a10e-4cbc-a4df-19d6c5f22569 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f70bd60330a84833a2573fc32cde778b, user_id=a8aca9f149a5468f9207c5f42f724f05, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:27.833183 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c3577-a10e-4cbc-a4df-19d6c5f22569 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:27.874707 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1139/2288] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:47:27 2022] PUT /identity/v3/projects/f70bd60330a84833a2573fc32cde778b/users/a8aca9f149a5468f9207c5f42f724f05/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:47:27.883032 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4a83d-2581-4f31-b201-d78dfc6efae3 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:27.883680 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4a83d-2581-4f31-b201-d78dfc6efae3 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:27.884398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4a83d-2581-4f31-b201-d78dfc6efae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:27.924793 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-49c4a83d-2581-4f31-b201-d78dfc6efae3 None None] MFA Rules not processed for user `a8aca9f149a5468f9207c5f42f724f05`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:47:27.926110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-49c4a83d-2581-4f31-b201-d78dfc6efae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:27.956437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:47:28.006112 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1150/2289] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:47:27 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:47:28.018556 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c3577-a10e-4cbc-a4df-19d6c5f22569 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:28.020517 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1e4c3577-a10e-4cbc-a4df-19d6c5f22569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:28.044245 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:47:28.106319 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c3577-a10e-4cbc-a4df-19d6c5f22569 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:28.113325 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf9ac6-3fab-4359-8ef4-b9c4ebfdf99a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:28.114045 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf9ac6-3fab-4359-8ef4-b9c4ebfdf99a service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:28.114513 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf9ac6-3fab-4359-8ef4-b9c4ebfdf99a service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:28.114863 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf9ac6-3fab-4359-8ef4-b9c4ebfdf99a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:28.126398 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf9ac6-3fab-4359-8ef4-b9c4ebfdf99a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:28.127853 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf9ac6-3fab-4359-8ef4-b9c4ebfdf99a service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:28.139726 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1140/2290] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:47:28 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:47:28.332659 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4a83d-2581-4f31-b201-d78dfc6efae3 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:28.344210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4a83d-2581-4f31-b201-d78dfc6efae3 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:28.348042 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d51912-d503-4e6c-be47-44ae3d49faa9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:28.348652 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d51912-d503-4e6c-be47-44ae3d49faa9 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:28.348930 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d51912-d503-4e6c-be47-44ae3d49faa9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:28.349244 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d51912-d503-4e6c-be47-44ae3d49faa9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:28.424545 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d51912-d503-4e6c-be47-44ae3d49faa9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:28.424545 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d51912-d503-4e6c-be47-44ae3d49faa9 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:28.434855 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1151/2291] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:47:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:47:31.881793 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf9ac6-3fab-4359-8ef4-b9c4ebfdf99a service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:31.943182 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf9ac6-3fab-4359-8ef4-b9c4ebfdf99a service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:31.947786 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a2e9b-5abf-4a38-a28e-54ffe0af5482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:31.948487 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a2e9b-5abf-4a38-a28e-54ffe0af5482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:31.949017 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a2e9b-5abf-4a38-a28e-54ffe0af5482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:31.949325 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a2e9b-5abf-4a38-a28e-54ffe0af5482 admin admin] PATH_INFO: `/v3/users/1a95dc87e62c4ac2a74bf7e82a8fae79` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:31.950464 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a2e9b-5abf-4a38-a28e-54ffe0af5482 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a95dc87e62c4ac2a74bf7e82a8fae79)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:31.952408 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a2e9b-5abf-4a38-a28e-54ffe0af5482 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:32.117298 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-0f6a2e9b-5abf-4a38-a28e-54ffe0af5482 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a95dc87e62c4ac2a74bf7e82a8fae79'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:32.117905 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-0f6a2e9b-5abf-4a38-a28e-54ffe0af5482 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a95dc87e62c4ac2a74bf7e82a8fae79'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:32.118531 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-0f6a2e9b-5abf-4a38-a28e-54ffe0af5482 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a95dc87e62c4ac2a74bf7e82a8fae79'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:32.126535 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-0f6a2e9b-5abf-4a38-a28e-54ffe0af5482 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a95dc87e62c4ac2a74bf7e82a8fae79'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:32.192819 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-0f6a2e9b-5abf-4a38-a28e-54ffe0af5482 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a95dc87e62c4ac2a74bf7e82a8fae79'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:32.212433 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1141/2292] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:47:31 2022] DELETE /identity/v3/users/1a95dc87e62c4ac2a74bf7e82a8fae79 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:47:32.218170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabdfb3-c2e6-4ef9-85fe-9d63c7d851c5 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:32.218665 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabdfb3-c2e6-4ef9-85fe-9d63c7d851c5 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:32.218779 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabdfb3-c2e6-4ef9-85fe-9d63c7d851c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:32.238767 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-daabdfb3-c2e6-4ef9-85fe-9d63c7d851c5 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:47:32.239958 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-daabdfb3-c2e6-4ef9-85fe-9d63c7d851c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:32.302567 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1152/2293] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:47:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:47:32.313156 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a2e9b-5abf-4a38-a28e-54ffe0af5482 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:32.314788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0f6a2e9b-5abf-4a38-a28e-54ffe0af5482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:32.361396 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:47:32.376893 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a2e9b-5abf-4a38-a28e-54ffe0af5482 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:32.380778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0b4b9-6d91-4e72-8c1f-f3f2cf259d68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:32.381432 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0b4b9-6d91-4e72-8c1f-f3f2cf259d68 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:32.381710 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0b4b9-6d91-4e72-8c1f-f3f2cf259d68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:32.381980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0b4b9-6d91-4e72-8c1f-f3f2cf259d68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:32.391555 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0b4b9-6d91-4e72-8c1f-f3f2cf259d68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:32.392990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0b4b9-6d91-4e72-8c1f-f3f2cf259d68 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:32.403495 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1142/2294] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:47:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:47:32.864137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabdfb3-c2e6-4ef9-85fe-9d63c7d851c5 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:32.865754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-daabdfb3-c2e6-4ef9-85fe-9d63c7d851c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:32.879269 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabdfb3-c2e6-4ef9-85fe-9d63c7d851c5 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:32.884934 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9093ac65-890e-474a-b620-a2391f64b525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:32.885684 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093ac65-890e-474a-b620-a2391f64b525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:32.886196 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093ac65-890e-474a-b620-a2391f64b525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:32.886540 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093ac65-890e-474a-b620-a2391f64b525 admin admin] PATH_INFO: `/v3/projects/9e0ed574f00e475c9af0e464895c4845` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:32.888155 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9093ac65-890e-474a-b620-a2391f64b525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e0ed574f00e475c9af0e464895c4845)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:32.889983 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9093ac65-890e-474a-b620-a2391f64b525 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:33.589322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9093ac65-890e-474a-b620-a2391f64b525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e0ed574f00e475c9af0e464895c4845', 'request_id': 'req-9093ac65-890e-474a-b620-a2391f64b525'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:33.844291 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9093ac65-890e-474a-b620-a2391f64b525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e0ed574f00e475c9af0e464895c4845', 'request_id': 'req-9093ac65-890e-474a-b620-a2391f64b525'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:33.845970 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:47:33.862918 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9093ac65-890e-474a-b620-a2391f64b525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e0ed574f00e475c9af0e464895c4845', 'request_id': 'req-9093ac65-890e-474a-b620-a2391f64b525'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:33.873654 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9093ac65-890e-474a-b620-a2391f64b525 admin admin] The token cache is being invalidate because project 9e0ed574f00e475c9af0e464895c4845 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:47:33.873941 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9093ac65-890e-474a-b620-a2391f64b525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:33.875121 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9093ac65-890e-474a-b620-a2391f64b525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:33.876627 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1153/2295] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:47:32 2022] DELETE /identity/v3/projects/9e0ed574f00e475c9af0e464895c4845 => generated 0 bytes in 1013 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:47:33.880604 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0b4b9-6d91-4e72-8c1f-f3f2cf259d68 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:33.891047 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0b4b9-6d91-4e72-8c1f-f3f2cf259d68 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:33.948096 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1ab23-0f51-4812-abaf-f92a7ad01dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:33.948739 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1ab23-0f51-4812-abaf-f92a7ad01dd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:33.949018 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1ab23-0f51-4812-abaf-f92a7ad01dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:33.949606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1ab23-0f51-4812-abaf-f92a7ad01dd1 admin admin] PATH_INFO: `/v3/users/a8aca9f149a5468f9207c5f42f724f05` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:33.950795 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc1ab23-0f51-4812-abaf-f92a7ad01dd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8aca9f149a5468f9207c5f42f724f05)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:33.953952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc1ab23-0f51-4812-abaf-f92a7ad01dd1 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:34.127126 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-fcc1ab23-0f51-4812-abaf-f92a7ad01dd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8aca9f149a5468f9207c5f42f724f05'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:34.127833 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-fcc1ab23-0f51-4812-abaf-f92a7ad01dd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8aca9f149a5468f9207c5f42f724f05'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:34.128485 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-fcc1ab23-0f51-4812-abaf-f92a7ad01dd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8aca9f149a5468f9207c5f42f724f05'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:34.135530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-fcc1ab23-0f51-4812-abaf-f92a7ad01dd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8aca9f149a5468f9207c5f42f724f05'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:34.210331 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-fcc1ab23-0f51-4812-abaf-f92a7ad01dd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8aca9f149a5468f9207c5f42f724f05'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:34.229464 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1143/2296] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:47:33 2022] DELETE /identity/v3/users/a8aca9f149a5468f9207c5f42f724f05 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:47:34.365107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9093ac65-890e-474a-b620-a2391f64b525 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:34.366704 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9093ac65-890e-474a-b620-a2391f64b525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:34.472479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9093ac65-890e-474a-b620-a2391f64b525 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:34.586913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d93e80-445c-4b50-853d-895e0672465f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:34.587821 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d93e80-445c-4b50-853d-895e0672465f service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:34.588642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d93e80-445c-4b50-853d-895e0672465f service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:34.593524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d93e80-445c-4b50-853d-895e0672465f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:34.595534 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-09d93e80-445c-4b50-853d-895e0672465f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:34.607794 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d93e80-445c-4b50-853d-895e0672465f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:34.609698 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d93e80-445c-4b50-853d-895e0672465f service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:34.620039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1154/2297] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:47:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:47:34.675323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1ab23-0f51-4812-abaf-f92a7ad01dd1 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:34.677604 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-fcc1ab23-0f51-4812-abaf-f92a7ad01dd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:34.771707 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1ab23-0f51-4812-abaf-f92a7ad01dd1 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:34.780589 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785cdeb-0e31-432b-9228-163ecd0e8d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:34.781466 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785cdeb-0e31-432b-9228-163ecd0e8d54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:34.781742 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785cdeb-0e31-432b-9228-163ecd0e8d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:34.782016 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785cdeb-0e31-432b-9228-163ecd0e8d54 admin admin] PATH_INFO: `/v3/projects/f70bd60330a84833a2573fc32cde778b` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:34.786556 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785cdeb-0e31-432b-9228-163ecd0e8d54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f70bd60330a84833a2573fc32cde778b)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:34.791880 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785cdeb-0e31-432b-9228-163ecd0e8d54 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:34.791880 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:47:34.957599 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-3785cdeb-0e31-432b-9228-163ecd0e8d54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f70bd60330a84833a2573fc32cde778b', 'request_id': 'req-3785cdeb-0e31-432b-9228-163ecd0e8d54'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:34.964086 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-3785cdeb-0e31-432b-9228-163ecd0e8d54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f70bd60330a84833a2573fc32cde778b', 'request_id': 'req-3785cdeb-0e31-432b-9228-163ecd0e8d54'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:34.970080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-3785cdeb-0e31-432b-9228-163ecd0e8d54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f70bd60330a84833a2573fc32cde778b', 'request_id': 'req-3785cdeb-0e31-432b-9228-163ecd0e8d54'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:35.067319 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-3785cdeb-0e31-432b-9228-163ecd0e8d54 admin admin] The token cache is being invalidate because project f70bd60330a84833a2573fc32cde778b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:47:35.071913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-3785cdeb-0e31-432b-9228-163ecd0e8d54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:35.071913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-3785cdeb-0e31-432b-9228-163ecd0e8d54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:35.071913 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1144/2298] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:47:34 2022] DELETE /identity/v3/projects/f70bd60330a84833a2573fc32cde778b => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:47:35.074762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4b56e-40f2-4ad0-9486-2fb53c400ce3 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:35.075635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4b56e-40f2-4ad0-9486-2fb53c400ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:35.075982 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4b56e-40f2-4ad0-9486-2fb53c400ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:35.100784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-bbc4b56e-40f2-4ad0-9486-2fb53c400ce3 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:47:35.102376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bbc4b56e-40f2-4ad0-9486-2fb53c400ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:35.164570 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1155/2299] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:47:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:47:35.172655 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736cf02-acd9-4394-a2d5-4a513eb108bd None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:35.172935 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736cf02-acd9-4394-a2d5-4a513eb108bd None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:35.173244 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736cf02-acd9-4394-a2d5-4a513eb108bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:35.193701 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-d736cf02-acd9-4394-a2d5-4a513eb108bd None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:47:35.194907 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d736cf02-acd9-4394-a2d5-4a513eb108bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:35.206671 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1145/2300] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:47:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:47:35.207880 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc4b56e-40f2-4ad0-9486-2fb53c400ce3 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:35.219330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc4b56e-40f2-4ad0-9486-2fb53c400ce3 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:35.223666 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d7630-be21-4228-aa95-c6267a654b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:35.224303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d7630-be21-4228-aa95-c6267a654b13 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:35.224581 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d7630-be21-4228-aa95-c6267a654b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:35.224848 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d7630-be21-4228-aa95-c6267a654b13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:35.225533 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d7630-be21-4228-aa95-c6267a654b13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:47:35.225833 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d7630-be21-4228-aa95-c6267a654b13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:35.227488 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d7630-be21-4228-aa95-c6267a654b13 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:35.230389 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:47:35.241965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1156/2301] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:47:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:47:35.246639 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736cf02-acd9-4394-a2d5-4a513eb108bd None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:35.256635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736cf02-acd9-4394-a2d5-4a513eb108bd None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:35.261634 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6cb79b-b5ba-42e1-a24f-ed9503cc8739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:35.262231 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cb79b-b5ba-42e1-a24f-ed9503cc8739 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:35.262504 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cb79b-b5ba-42e1-a24f-ed9503cc8739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:35.262783 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cb79b-b5ba-42e1-a24f-ed9503cc8739 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:35.263408 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6cb79b-b5ba-42e1-a24f-ed9503cc8739 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:35.265875 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6cb79b-b5ba-42e1-a24f-ed9503cc8739 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:35.345184 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1146/2302] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:47:35 2022] POST /identity/v3/projects => generated 409 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:47:35.350337 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d7630-be21-4228-aa95-c6267a654b13 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:35.362577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d7630-be21-4228-aa95-c6267a654b13 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:35.424417 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6fd8a5-6a37-4d80-9b07-f645ea32cea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:35.432420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fd8a5-6a37-4d80-9b07-f645ea32cea5 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:35.432420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fd8a5-6a37-4d80-9b07-f645ea32cea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:35.432420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fd8a5-6a37-4d80-9b07-f645ea32cea5 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:35.432420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6fd8a5-6a37-4d80-9b07-f645ea32cea5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:35.432420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6fd8a5-6a37-4d80-9b07-f645ea32cea5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:35.513450 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1157/2303] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:47:35 2022] POST /identity/v3/users => generated 305 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:47:35.517605 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6cb79b-b5ba-42e1-a24f-ed9503cc8739 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:35.528016 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6cb79b-b5ba-42e1-a24f-ed9503cc8739 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:35.533591 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c876e03-8113-4c95-b580-9e9a38539f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:35.534205 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c876e03-8113-4c95-b580-9e9a38539f55 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:35.534465 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c876e03-8113-4c95-b580-9e9a38539f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:35.534736 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c876e03-8113-4c95-b580-9e9a38539f55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:35.535373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c876e03-8113-4c95-b580-9e9a38539f55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:35.536949 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c876e03-8113-4c95-b580-9e9a38539f55 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:35.546815 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1147/2304] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:47:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:47:35.550902 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6fd8a5-6a37-4d80-9b07-f645ea32cea5 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:35.561226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6fd8a5-6a37-4d80-9b07-f645ea32cea5 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:35.568352 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c639f2-8f61-4a12-a68a-81c2f5d05238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:35.569036 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c639f2-8f61-4a12-a68a-81c2f5d05238 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:35.569358 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c639f2-8f61-4a12-a68a-81c2f5d05238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:35.570026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c639f2-8f61-4a12-a68a-81c2f5d05238 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:35.570466 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c639f2-8f61-4a12-a68a-81c2f5d05238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:35.572069 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c639f2-8f61-4a12-a68a-81c2f5d05238 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:35.588036 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1158/2305] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:47:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:47:35.589425 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c876e03-8113-4c95-b580-9e9a38539f55 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:35.600524 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c876e03-8113-4c95-b580-9e9a38539f55 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:35.605439 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e8ff94-9e18-4b61-95dc-98ed9fc42b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:35.606084 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8ff94-9e18-4b61-95dc-98ed9fc42b46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:35.606365 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8ff94-9e18-4b61-95dc-98ed9fc42b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:35.606636 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8ff94-9e18-4b61-95dc-98ed9fc42b46 admin admin] PATH_INFO: `/v3/projects/bf080fea2d8f475a85baa1d1b75c02c4/users/c13d4cb295964913812bb871b7ccdfab/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:35.621103 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e8ff94-9e18-4b61-95dc-98ed9fc42b46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf080fea2d8f475a85baa1d1b75c02c4, user_id=c13d4cb295964913812bb871b7ccdfab, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:35.623769 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e8ff94-9e18-4b61-95dc-98ed9fc42b46 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:35.679434 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1148/2306] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:47:35 2022] PUT /identity/v3/projects/bf080fea2d8f475a85baa1d1b75c02c4/users/c13d4cb295964913812bb871b7ccdfab/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:47:35.690839 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca8909-787f-40a5-8e29-fc708e764e95 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:35.691356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca8909-787f-40a5-8e29-fc708e764e95 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:35.691470 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca8909-787f-40a5-8e29-fc708e764e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:35.720242 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8cca8909-787f-40a5-8e29-fc708e764e95 None None] MFA Rules not processed for user `c13d4cb295964913812bb871b7ccdfab`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:47:35.721584 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8cca8909-787f-40a5-8e29-fc708e764e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:35.787125 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1159/2307] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:47:35 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:47:35.791857 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e8ff94-9e18-4b61-95dc-98ed9fc42b46 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:35.854580 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e8ff94-9e18-4b61-95dc-98ed9fc42b46 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:35.859448 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443d59c-976f-4b64-9451-dbed0004ffa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:35.860054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443d59c-976f-4b64-9451-dbed0004ffa6 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:35.860341 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443d59c-976f-4b64-9451-dbed0004ffa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:35.860614 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443d59c-976f-4b64-9451-dbed0004ffa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:35.861307 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443d59c-976f-4b64-9451-dbed0004ffa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:35.863181 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443d59c-976f-4b64-9451-dbed0004ffa6 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:35.938153 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1149/2308] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:47:35 2022] POST /identity/v3/projects => generated 409 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:47:35.942515 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca8909-787f-40a5-8e29-fc708e764e95 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:35.953586 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca8909-787f-40a5-8e29-fc708e764e95 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:36.010712 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6aa5ee-1f92-45b2-80ce-aeb421a90aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:36.011323 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6aa5ee-1f92-45b2-80ce-aeb421a90aef admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:36.011603 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6aa5ee-1f92-45b2-80ce-aeb421a90aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:36.011878 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6aa5ee-1f92-45b2-80ce-aeb421a90aef admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:36.012501 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6aa5ee-1f92-45b2-80ce-aeb421a90aef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:36.014534 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6aa5ee-1f92-45b2-80ce-aeb421a90aef admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:36.024141 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:47:36.113786 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1160/2309] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:47:35 2022] POST /identity/v3/users => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:47:36.118188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443d59c-976f-4b64-9451-dbed0004ffa6 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:36.128606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443d59c-976f-4b64-9451-dbed0004ffa6 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:36.133287 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0939684-a8f4-4c11-a7f6-c33589195a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:36.133894 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0939684-a8f4-4c11-a7f6-c33589195a23 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:36.134170 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0939684-a8f4-4c11-a7f6-c33589195a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:36.134440 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0939684-a8f4-4c11-a7f6-c33589195a23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:36.135087 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0939684-a8f4-4c11-a7f6-c33589195a23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:36.136659 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0939684-a8f4-4c11-a7f6-c33589195a23 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:36.142143 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:47:36.147793 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1150/2310] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:47:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:47:36.158296 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6aa5ee-1f92-45b2-80ce-aeb421a90aef admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:36.169206 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6aa5ee-1f92-45b2-80ce-aeb421a90aef admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:36.175065 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3eedd2-a5b2-472a-81a7-82ae27c993d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:36.175738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3eedd2-a5b2-472a-81a7-82ae27c993d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:36.176021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3eedd2-a5b2-472a-81a7-82ae27c993d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:36.177081 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3eedd2-a5b2-472a-81a7-82ae27c993d5 admin admin] PATH_INFO: `/v3/projects/3b09f7ade1a24abca74d60c15f31a708/users/5f134c44255642dbaecde82fddc856e5/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:36.192734 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3eedd2-a5b2-472a-81a7-82ae27c993d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b09f7ade1a24abca74d60c15f31a708, user_id=5f134c44255642dbaecde82fddc856e5, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:36.195446 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3eedd2-a5b2-472a-81a7-82ae27c993d5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:36.273618 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1161/2311] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:47:36 2022] PUT /identity/v3/projects/3b09f7ade1a24abca74d60c15f31a708/users/5f134c44255642dbaecde82fddc856e5/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:47:36.277367 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0939684-a8f4-4c11-a7f6-c33589195a23 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:36.289686 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0939684-a8f4-4c11-a7f6-c33589195a23 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:36.345935 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77800857-6f65-4222-9f56-1c25dbee479d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:36.346542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77800857-6f65-4222-9f56-1c25dbee479d admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:36.346824 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77800857-6f65-4222-9f56-1c25dbee479d admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:36.352024 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77800857-6f65-4222-9f56-1c25dbee479d admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:36.352024 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77800857-6f65-4222-9f56-1c25dbee479d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:36.352024 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77800857-6f65-4222-9f56-1c25dbee479d admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:36.359431 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1151/2312] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:47:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:47:36.363756 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3eedd2-a5b2-472a-81a7-82ae27c993d5 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:36.375361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3eedd2-a5b2-472a-81a7-82ae27c993d5 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:36.379141 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a074f84-0849-4899-83c0-f98666ae7aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:36.379804 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a074f84-0849-4899-83c0-f98666ae7aa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:36.380114 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a074f84-0849-4899-83c0-f98666ae7aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:36.380388 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a074f84-0849-4899-83c0-f98666ae7aa7 admin admin] PATH_INFO: `/v3/projects/3b09f7ade1a24abca74d60c15f31a708/users/5f134c44255642dbaecde82fddc856e5/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:36.382456 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a074f84-0849-4899-83c0-f98666ae7aa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b09f7ade1a24abca74d60c15f31a708, user_id=5f134c44255642dbaecde82fddc856e5, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:36.385028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a074f84-0849-4899-83c0-f98666ae7aa7 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:36.396563 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1162/2313] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:47:36 2022] PUT /identity/v3/projects/3b09f7ade1a24abca74d60c15f31a708/users/5f134c44255642dbaecde82fddc856e5/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:47:36.407928 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77bf02a-d4d2-4a2d-a4ea-a31fbbcae386 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:36.408569 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77bf02a-d4d2-4a2d-a4ea-a31fbbcae386 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:36.408973 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77bf02a-d4d2-4a2d-a4ea-a31fbbcae386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:36.437775 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-c77bf02a-d4d2-4a2d-a4ea-a31fbbcae386 None None] MFA Rules not processed for user `5f134c44255642dbaecde82fddc856e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:47:36.438999 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c77bf02a-d4d2-4a2d-a4ea-a31fbbcae386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:36.509773 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1152/2314] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:47:36 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:47:36.522942 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a074f84-0849-4899-83c0-f98666ae7aa7 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:36.524663 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5a074f84-0849-4899-83c0-f98666ae7aa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:36.581448 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a074f84-0849-4899-83c0-f98666ae7aa7 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:36.585300 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f6d886-db3b-412a-bbb6-5ae273cacf85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:36.585893 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6d886-db3b-412a-bbb6-5ae273cacf85 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:36.586169 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6d886-db3b-412a-bbb6-5ae273cacf85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:36.586440 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6d886-db3b-412a-bbb6-5ae273cacf85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:36.596063 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f6d886-db3b-412a-bbb6-5ae273cacf85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:36.597526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f6d886-db3b-412a-bbb6-5ae273cacf85 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:36.607646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1163/2315] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:47:36 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:47:36.636754 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ee4b8-6df7-42c6-a874-ac0973a733d9 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:36.637044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ee4b8-6df7-42c6-a874-ac0973a733d9 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:36.637360 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ee4b8-6df7-42c6-a874-ac0973a733d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:36.665783 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-1b1ee4b8-6df7-42c6-a874-ac0973a733d9 None None] MFA Rules not processed for user `7f3422c6f9a041ceba851b71bd15d0df`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:47:36.666981 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1b1ee4b8-6df7-42c6-a874-ac0973a733d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:36.735684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1153/2316] 10.222.0.33 () {58 vars in 967 bytes} [Mon Aug 8 08:47:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:47:36.748110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f6d886-db3b-412a-bbb6-5ae273cacf85 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:36.750294 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-66f6d886-db3b-412a-bbb6-5ae273cacf85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:36.820384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f6d886-db3b-412a-bbb6-5ae273cacf85 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:36.825524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069d7a3-ed2e-4e7d-967c-914790261ace service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e69c0e02ad944f2a85fc5318018e5cb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:36.826132 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069d7a3-ed2e-4e7d-967c-914790261ace service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:36.826437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069d7a3-ed2e-4e7d-967c-914790261ace service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:36.826711 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069d7a3-ed2e-4e7d-967c-914790261ace service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:36.835777 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069d7a3-ed2e-4e7d-967c-914790261ace service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:36.837394 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069d7a3-ed2e-4e7d-967c-914790261ace service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:36.845528 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1164/2317] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 08:47:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 08:47:36.938251 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ee4b8-6df7-42c6-a874-ac0973a733d9 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:36.939677 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1b1ee4b8-6df7-42c6-a874-ac0973a733d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:36.954565 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ee4b8-6df7-42c6-a874-ac0973a733d9 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:36.960742 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b496c35-c8db-464c-87da-a03540171663 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:36.961677 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b496c35-c8db-464c-87da-a03540171663 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:36.962195 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b496c35-c8db-464c-87da-a03540171663 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:36.962826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b496c35-c8db-464c-87da-a03540171663 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:37.009287 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:47:37.066906 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b496c35-c8db-464c-87da-a03540171663 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:37.069259 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b496c35-c8db-464c-87da-a03540171663 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:37.078948 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1154/2318] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:47:36 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:47:46.842423 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:INSERT=3,keystone:SELECT=75 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:47:46.926715 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069d7a3-ed2e-4e7d-967c-914790261ace service swift] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:46.989032 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069d7a3-ed2e-4e7d-967c-914790261ace service swift] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:46.993693 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131a341-d378-48e0-ac33-9b58c921420a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:46.994327 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131a341-d378-48e0-ac33-9b58c921420a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:46.994604 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131a341-d378-48e0-ac33-9b58c921420a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:46.994879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131a341-d378-48e0-ac33-9b58c921420a admin admin] PATH_INFO: `/v3/users/c13d4cb295964913812bb871b7ccdfab` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:46.996000 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5131a341-d378-48e0-ac33-9b58c921420a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c13d4cb295964913812bb871b7ccdfab)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:46.998021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5131a341-d378-48e0-ac33-9b58c921420a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:47.096309 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=16 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:47:47.206703 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5131a341-d378-48e0-ac33-9b58c921420a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c13d4cb295964913812bb871b7ccdfab'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:47.207427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5131a341-d378-48e0-ac33-9b58c921420a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c13d4cb295964913812bb871b7ccdfab'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:47.208393 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5131a341-d378-48e0-ac33-9b58c921420a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c13d4cb295964913812bb871b7ccdfab'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:47.214825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5131a341-d378-48e0-ac33-9b58c921420a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c13d4cb295964913812bb871b7ccdfab'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:47.278116 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-5131a341-d378-48e0-ac33-9b58c921420a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c13d4cb295964913812bb871b7ccdfab'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:47.297570 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1165/2319] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:47:46 2022] DELETE /identity/v3/users/c13d4cb295964913812bb871b7ccdfab => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:47:47.302066 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcfdcf7-dbae-4693-9d59-3f0ef59d0c90 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:47.302349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcfdcf7-dbae-4693-9d59-3f0ef59d0c90 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:47.302620 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcfdcf7-dbae-4693-9d59-3f0ef59d0c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:47.320952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-2dcfdcf7-dbae-4693-9d59-3f0ef59d0c90 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:47:47.323248 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2dcfdcf7-dbae-4693-9d59-3f0ef59d0c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:47.378550 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1155/2320] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:47:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:47:47.388028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131a341-d378-48e0-ac33-9b58c921420a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:47.389590 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5131a341-d378-48e0-ac33-9b58c921420a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:47.446661 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131a341-d378-48e0-ac33-9b58c921420a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:47.450259 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6fafe7-2f72-4137-a226-d102b167e256 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:47.450884 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fafe7-2f72-4137-a226-d102b167e256 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:47.451161 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fafe7-2f72-4137-a226-d102b167e256 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:47.451915 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6fafe7-2f72-4137-a226-d102b167e256 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:47.461622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6fafe7-2f72-4137-a226-d102b167e256 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:47.463126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6fafe7-2f72-4137-a226-d102b167e256 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:47.473403 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1166/2321] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:47:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:47:47.880332 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcfdcf7-dbae-4693-9d59-3f0ef59d0c90 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:47.881876 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2dcfdcf7-dbae-4693-9d59-3f0ef59d0c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:47.895646 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcfdcf7-dbae-4693-9d59-3f0ef59d0c90 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:47.900506 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85042b2c-0693-43b8-ba76-5d85034a7192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:47.901319 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85042b2c-0693-43b8-ba76-5d85034a7192 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:47.901696 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85042b2c-0693-43b8-ba76-5d85034a7192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:47.902071 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85042b2c-0693-43b8-ba76-5d85034a7192 admin admin] PATH_INFO: `/v3/projects/bf080fea2d8f475a85baa1d1b75c02c4` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:47.903319 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85042b2c-0693-43b8-ba76-5d85034a7192 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf080fea2d8f475a85baa1d1b75c02c4)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:47.906071 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85042b2c-0693-43b8-ba76-5d85034a7192 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:48.005355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-85042b2c-0693-43b8-ba76-5d85034a7192 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf080fea2d8f475a85baa1d1b75c02c4', 'request_id': 'req-85042b2c-0693-43b8-ba76-5d85034a7192'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:48.011716 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-85042b2c-0693-43b8-ba76-5d85034a7192 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf080fea2d8f475a85baa1d1b75c02c4', 'request_id': 'req-85042b2c-0693-43b8-ba76-5d85034a7192'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:48.017881 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-85042b2c-0693-43b8-ba76-5d85034a7192 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf080fea2d8f475a85baa1d1b75c02c4', 'request_id': 'req-85042b2c-0693-43b8-ba76-5d85034a7192'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:48.107498 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-85042b2c-0693-43b8-ba76-5d85034a7192 admin admin] The token cache is being invalidate because project bf080fea2d8f475a85baa1d1b75c02c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:47:48.108782 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-85042b2c-0693-43b8-ba76-5d85034a7192 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:48.109353 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-85042b2c-0693-43b8-ba76-5d85034a7192 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:48.110869 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1156/2322] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:47:47 2022] DELETE /identity/v3/projects/bf080fea2d8f475a85baa1d1b75c02c4 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:47:48.114726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6fafe7-2f72-4137-a226-d102b167e256 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:48.125011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6fafe7-2f72-4137-a226-d102b167e256 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:48.142961 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:47:48.181058 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2087c-4426-4272-beab-eb336354c97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:48.181753 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2087c-4426-4272-beab-eb336354c97c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:48.182110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2087c-4426-4272-beab-eb336354c97c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:48.182327 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2087c-4426-4272-beab-eb336354c97c admin admin] PATH_INFO: `/v3/users/5f134c44255642dbaecde82fddc856e5` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:48.183432 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec2087c-4426-4272-beab-eb336354c97c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f134c44255642dbaecde82fddc856e5)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:48.185509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec2087c-4426-4272-beab-eb336354c97c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:48.343508 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-aec2087c-4426-4272-beab-eb336354c97c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f134c44255642dbaecde82fddc856e5'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:48.344149 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-aec2087c-4426-4272-beab-eb336354c97c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f134c44255642dbaecde82fddc856e5'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:48.344740 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-aec2087c-4426-4272-beab-eb336354c97c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f134c44255642dbaecde82fddc856e5'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:48.351551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-aec2087c-4426-4272-beab-eb336354c97c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f134c44255642dbaecde82fddc856e5'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:48.438362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-aec2087c-4426-4272-beab-eb336354c97c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f134c44255642dbaecde82fddc856e5'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:48.456954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1167/2323] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:47:48 2022] DELETE /identity/v3/users/5f134c44255642dbaecde82fddc856e5 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:47:48.898368 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85042b2c-0693-43b8-ba76-5d85034a7192 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:48.900056 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-85042b2c-0693-43b8-ba76-5d85034a7192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:48.915127 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85042b2c-0693-43b8-ba76-5d85034a7192 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:48.972457 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29205c4e-c07a-45fa-b757-502f537031fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:48.973013 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29205c4e-c07a-45fa-b757-502f537031fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:48.973325 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29205c4e-c07a-45fa-b757-502f537031fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:48.973627 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29205c4e-c07a-45fa-b757-502f537031fb admin admin] PATH_INFO: `/v3/projects/3b09f7ade1a24abca74d60c15f31a708` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:48.974735 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29205c4e-c07a-45fa-b757-502f537031fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b09f7ade1a24abca74d60c15f31a708)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:48.976633 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29205c4e-c07a-45fa-b757-502f537031fb admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:49.080004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:47:49.096368 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-29205c4e-c07a-45fa-b757-502f537031fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b09f7ade1a24abca74d60c15f31a708', 'request_id': 'req-29205c4e-c07a-45fa-b757-502f537031fb'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:49.103053 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-29205c4e-c07a-45fa-b757-502f537031fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b09f7ade1a24abca74d60c15f31a708', 'request_id': 'req-29205c4e-c07a-45fa-b757-502f537031fb'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:49.109818 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-29205c4e-c07a-45fa-b757-502f537031fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b09f7ade1a24abca74d60c15f31a708', 'request_id': 'req-29205c4e-c07a-45fa-b757-502f537031fb'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:49.187108 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-29205c4e-c07a-45fa-b757-502f537031fb admin admin] The token cache is being invalidate because project 3b09f7ade1a24abca74d60c15f31a708 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:47:49.187397 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-29205c4e-c07a-45fa-b757-502f537031fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:49.188049 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-29205c4e-c07a-45fa-b757-502f537031fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:47:49.189568 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1157/2324] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:47:48 2022] DELETE /identity/v3/projects/3b09f7ade1a24abca74d60c15f31a708 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:47:49.194574 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248be79-6945-48df-8f54-71408c2315cf None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:49.195073 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248be79-6945-48df-8f54-71408c2315cf None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:49.195172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248be79-6945-48df-8f54-71408c2315cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:49.214911 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-f248be79-6945-48df-8f54-71408c2315cf None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:47:49.216031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f248be79-6945-48df-8f54-71408c2315cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:49.275207 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1168/2325] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:47:49.283994 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a042f-bf44-48f0-9b30-9983656b613b None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:49.284282 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a042f-bf44-48f0-9b30-9983656b613b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:49.284557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a042f-bf44-48f0-9b30-9983656b613b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:49.303906 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-902a042f-bf44-48f0-9b30-9983656b613b None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:47:49.305094 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-902a042f-bf44-48f0-9b30-9983656b613b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:49.313283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1158/2326] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:47:49.317776 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f248be79-6945-48df-8f54-71408c2315cf None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:49.328351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f248be79-6945-48df-8f54-71408c2315cf None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:49.332992 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee59ca-4b90-4e10-8ac5-dfb21ed65144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:49.333633 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee59ca-4b90-4e10-8ac5-dfb21ed65144 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:49.333913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee59ca-4b90-4e10-8ac5-dfb21ed65144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:49.334251 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee59ca-4b90-4e10-8ac5-dfb21ed65144 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:49.334929 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee59ca-4b90-4e10-8ac5-dfb21ed65144 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:47:49.335253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee59ca-4b90-4e10-8ac5-dfb21ed65144 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:49.336854 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee59ca-4b90-4e10-8ac5-dfb21ed65144 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:49.350581 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1169/2327] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:47:49.354034 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902a042f-bf44-48f0-9b30-9983656b613b None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:49.363354 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902a042f-bf44-48f0-9b30-9983656b613b None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:49.367890 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef409a46-977b-4469-b471-008ce71bd230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:49.368477 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef409a46-977b-4469-b471-008ce71bd230 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:49.368756 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef409a46-977b-4469-b471-008ce71bd230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:49.369028 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef409a46-977b-4469-b471-008ce71bd230 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:49.369675 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef409a46-977b-4469-b471-008ce71bd230 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:49.371540 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef409a46-977b-4469-b471-008ce71bd230 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:49.468530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1159/2328] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:47:49 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:47:49.472842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee59ca-4b90-4e10-8ac5-dfb21ed65144 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:49.483364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee59ca-4b90-4e10-8ac5-dfb21ed65144 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:49.509024 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:47:49.539731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f5e35-35ca-41af-848e-0fc24c078887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:49.540571 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f5e35-35ca-41af-848e-0fc24c078887 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:49.541028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f5e35-35ca-41af-848e-0fc24c078887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:49.541806 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f5e35-35ca-41af-848e-0fc24c078887 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:49.542436 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f5e35-35ca-41af-848e-0fc24c078887 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:49.544423 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f5e35-35ca-41af-848e-0fc24c078887 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:49.627786 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1170/2329] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:47:49 2022] POST /identity/v3/users => generated 287 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:47:49.632297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef409a46-977b-4469-b471-008ce71bd230 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:49.643859 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef409a46-977b-4469-b471-008ce71bd230 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:49.648660 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fbd4f4-fb90-42eb-a2ba-34e9cd27eaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:49.649308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbd4f4-fb90-42eb-a2ba-34e9cd27eaf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:49.649587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbd4f4-fb90-42eb-a2ba-34e9cd27eaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:49.649854 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbd4f4-fb90-42eb-a2ba-34e9cd27eaf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:49.650498 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fbd4f4-fb90-42eb-a2ba-34e9cd27eaf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:49.652076 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fbd4f4-fb90-42eb-a2ba-34e9cd27eaf9 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:49.663110 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1160/2330] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:47:49.667303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f5e35-35ca-41af-848e-0fc24c078887 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:49.677851 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f5e35-35ca-41af-848e-0fc24c078887 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:49.683198 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14ca14-f654-483d-b625-35cbf8c0ff60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:49.683799 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14ca14-f654-483d-b625-35cbf8c0ff60 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:49.687472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14ca14-f654-483d-b625-35cbf8c0ff60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:49.687472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14ca14-f654-483d-b625-35cbf8c0ff60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:49.687472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14ca14-f654-483d-b625-35cbf8c0ff60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:49.687472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14ca14-f654-483d-b625-35cbf8c0ff60 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:49.704757 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1171/2331] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:47:49.705972 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fbd4f4-fb90-42eb-a2ba-34e9cd27eaf9 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:49.716933 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fbd4f4-fb90-42eb-a2ba-34e9cd27eaf9 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:49.721970 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac924322-13ba-460e-8cb2-381c7b32f473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:49.722667 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac924322-13ba-460e-8cb2-381c7b32f473 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:49.722940 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac924322-13ba-460e-8cb2-381c7b32f473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:49.723217 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac924322-13ba-460e-8cb2-381c7b32f473 admin admin] PATH_INFO: `/v3/projects/9537e903aa1646ee8f3774b7dd417309/users/a02c4fdcb5ad418d93e0629f604babea/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:49.740595 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac924322-13ba-460e-8cb2-381c7b32f473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9537e903aa1646ee8f3774b7dd417309, user_id=a02c4fdcb5ad418d93e0629f604babea, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:49.749407 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac924322-13ba-460e-8cb2-381c7b32f473 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:49.828443 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1161/2332] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:47:49 2022] PUT /identity/v3/projects/9537e903aa1646ee8f3774b7dd417309/users/a02c4fdcb5ad418d93e0629f604babea/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:47:49.835277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba1763-46f5-4c6a-9c02-730514c9673f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:49.835587 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba1763-46f5-4c6a-9c02-730514c9673f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:49.835862 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba1763-46f5-4c6a-9c02-730514c9673f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:49.855693 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b7ba1763-46f5-4c6a-9c02-730514c9673f None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:47:49.856942 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b7ba1763-46f5-4c6a-9c02-730514c9673f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:49.921011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1172/2333] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:47:49.937693 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac924322-13ba-460e-8cb2-381c7b32f473 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:49.939415 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ac924322-13ba-460e-8cb2-381c7b32f473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:50.005818 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac924322-13ba-460e-8cb2-381c7b32f473 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:50.009655 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f35e2-b9f5-4e13-b75d-779d227f36fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:50.010273 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f35e2-b9f5-4e13-b75d-779d227f36fa service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:50.010538 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f35e2-b9f5-4e13-b75d-779d227f36fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:50.010805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f35e2-b9f5-4e13-b75d-779d227f36fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:50.020154 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f35e2-b9f5-4e13-b75d-779d227f36fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:50.021610 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f35e2-b9f5-4e13-b75d-779d227f36fa service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:50.031217 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1162/2334] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:47:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:47:51.433355 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3f20d-0f20-4651-97a1-2a7670937b8c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:51.433893 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3f20d-0f20-4651-97a1-2a7670937b8c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:51.433893 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3f20d-0f20-4651-97a1-2a7670937b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:51.463593 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-29b3f20d-0f20-4651-97a1-2a7670937b8c None None] MFA Rules not processed for user `a02c4fdcb5ad418d93e0629f604babea`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:47:51.464812 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-29b3f20d-0f20-4651-97a1-2a7670937b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:51.535361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1173/2335] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:47:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:47:51.548228 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f35e2-b9f5-4e13-b75d-779d227f36fa service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:51.549667 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-801f35e2-b9f5-4e13-b75d-779d227f36fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:51.608173 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f35e2-b9f5-4e13-b75d-779d227f36fa service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:51.612173 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42877bf0-75e5-4ef2-b08f-4f04036e3375 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:51.613109 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42877bf0-75e5-4ef2-b08f-4f04036e3375 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:51.613460 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42877bf0-75e5-4ef2-b08f-4f04036e3375 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:51.613735 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42877bf0-75e5-4ef2-b08f-4f04036e3375 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:51.620430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:47:51.622985 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42877bf0-75e5-4ef2-b08f-4f04036e3375 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:51.625159 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42877bf0-75e5-4ef2-b08f-4f04036e3375 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:51.634276 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1163/2336] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:47:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:47:53.328292 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c594a61-432f-43e4-afb1-301367eeb86b None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:53.328292 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c594a61-432f-43e4-afb1-301367eeb86b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:53.328292 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c594a61-432f-43e4-afb1-301367eeb86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:53.349238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-9c594a61-432f-43e4-afb1-301367eeb86b None None] MFA Rules not processed for user `a02c4fdcb5ad418d93e0629f604babea`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:47:53.355107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-9c594a61-432f-43e4-afb1-301367eeb86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:53.363040 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1174/2337] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:47:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:47:53.632958 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42877bf0-75e5-4ef2-b08f-4f04036e3375 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:47:53.643764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42877bf0-75e5-4ef2-b08f-4f04036e3375 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:47:53.647416 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f3bd9-7901-4e4e-b768-b9ba9a774e36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:47:53.648011 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f3bd9-7901-4e4e-b768-b9ba9a774e36 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:47:53.648288 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f3bd9-7901-4e4e-b768-b9ba9a774e36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:47:53.648557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f3bd9-7901-4e4e-b768-b9ba9a774e36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:47:53.650455 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0e0f3bd9-7901-4e4e-b768-b9ba9a774e36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:47:53.661026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f3bd9-7901-4e4e-b768-b9ba9a774e36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:47:53.662478 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f3bd9-7901-4e4e-b768-b9ba9a774e36 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:47:53.671873 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1164/2338] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:47:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:48:00.515192 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c594a61-432f-43e4-afb1-301367eeb86b None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:00.524794 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c594a61-432f-43e4-afb1-301367eeb86b None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:00.528321 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ed8fb-8752-4e9b-b209-31bb2e380917 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:00.528920 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ed8fb-8752-4e9b-b209-31bb2e380917 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:00.529431 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ed8fb-8752-4e9b-b209-31bb2e380917 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:00.529705 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ed8fb-8752-4e9b-b209-31bb2e380917 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:00.531522 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-687ed8fb-8752-4e9b-b209-31bb2e380917 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:00.543809 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ed8fb-8752-4e9b-b209-31bb2e380917 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:00.545280 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ed8fb-8752-4e9b-b209-31bb2e380917 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:00.554949 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1175/2339] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:48:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:48:03.667125 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=8 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:04.342545 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f3bd9-7901-4e4e-b768-b9ba9a774e36 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:04.344518 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0e0f3bd9-7901-4e4e-b768-b9ba9a774e36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:04.446800 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f3bd9-7901-4e4e-b768-b9ba9a774e36 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:04.451801 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635bda3-dc35-41f5-9a39-02dad74834cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c23a782b9b4144bace1248c3b2e754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:04.452490 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635bda3-dc35-41f5-9a39-02dad74834cd service placement] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:04.452777 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635bda3-dc35-41f5-9a39-02dad74834cd service placement] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:04.453058 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635bda3-dc35-41f5-9a39-02dad74834cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:04.455904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-9635bda3-dc35-41f5-9a39-02dad74834cd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:04.468168 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635bda3-dc35-41f5-9a39-02dad74834cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:04.469794 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635bda3-dc35-41f5-9a39-02dad74834cd service placement] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:04.480764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1165/2340] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:48:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:48:04.518099 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ed8fb-8752-4e9b-b209-31bb2e380917 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:04.520137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-687ed8fb-8752-4e9b-b209-31bb2e380917 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:04.538117 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ed8fb-8752-4e9b-b209-31bb2e380917 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:04.542853 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214caaa6-ffc8-45e2-a0d8-f960039f12e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:04.543490 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214caaa6-ffc8-45e2-a0d8-f960039f12e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:04.543928 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214caaa6-ffc8-45e2-a0d8-f960039f12e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:04.544208 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214caaa6-ffc8-45e2-a0d8-f960039f12e1 admin admin] PATH_INFO: `/v3/users/32dcdc2e24e44a07ade44d1da330a77e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:04.545469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214caaa6-ffc8-45e2-a0d8-f960039f12e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32dcdc2e24e44a07ade44d1da330a77e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:04.547382 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214caaa6-ffc8-45e2-a0d8-f960039f12e1 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:04.562494 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:04.779607 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-214caaa6-ffc8-45e2-a0d8-f960039f12e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32dcdc2e24e44a07ade44d1da330a77e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:04.781319 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-214caaa6-ffc8-45e2-a0d8-f960039f12e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32dcdc2e24e44a07ade44d1da330a77e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:04.781948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-214caaa6-ffc8-45e2-a0d8-f960039f12e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32dcdc2e24e44a07ade44d1da330a77e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:04.789937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-214caaa6-ffc8-45e2-a0d8-f960039f12e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32dcdc2e24e44a07ade44d1da330a77e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:04.866233 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-214caaa6-ffc8-45e2-a0d8-f960039f12e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32dcdc2e24e44a07ade44d1da330a77e'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:04.884402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1176/2341] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:48:04 2022] DELETE /identity/v3/users/32dcdc2e24e44a07ade44d1da330a77e => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:05.371255 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635bda3-dc35-41f5-9a39-02dad74834cd service placement] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:05.383674 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635bda3-dc35-41f5-9a39-02dad74834cd service placement] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:05.457630 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77edf9ce-76f1-4e36-ae0b-ac70709d0fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:05.458202 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77edf9ce-76f1-4e36-ae0b-ac70709d0fd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:05.458468 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77edf9ce-76f1-4e36-ae0b-ac70709d0fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:05.458740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77edf9ce-76f1-4e36-ae0b-ac70709d0fd8 admin admin] PATH_INFO: `/v3/projects/ce958d94ddc441e9bd10b5ae9638accc` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:05.459967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77edf9ce-76f1-4e36-ae0b-ac70709d0fd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce958d94ddc441e9bd10b5ae9638accc)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:05.461996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77edf9ce-76f1-4e36-ae0b-ac70709d0fd8 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:05.565063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-77edf9ce-76f1-4e36-ae0b-ac70709d0fd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce958d94ddc441e9bd10b5ae9638accc', 'request_id': 'req-77edf9ce-76f1-4e36-ae0b-ac70709d0fd8'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:05.572272 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-77edf9ce-76f1-4e36-ae0b-ac70709d0fd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce958d94ddc441e9bd10b5ae9638accc', 'request_id': 'req-77edf9ce-76f1-4e36-ae0b-ac70709d0fd8'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:05.579146 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-77edf9ce-76f1-4e36-ae0b-ac70709d0fd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce958d94ddc441e9bd10b5ae9638accc', 'request_id': 'req-77edf9ce-76f1-4e36-ae0b-ac70709d0fd8'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:05.638231 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-77edf9ce-76f1-4e36-ae0b-ac70709d0fd8 admin admin] The token cache is being invalidate because project ce958d94ddc441e9bd10b5ae9638accc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:48:05.639487 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-77edf9ce-76f1-4e36-ae0b-ac70709d0fd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:05.640242 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-77edf9ce-76f1-4e36-ae0b-ac70709d0fd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:05.647090 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1166/2342] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:48:05 2022] DELETE /identity/v3/projects/ce958d94ddc441e9bd10b5ae9638accc => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:05.658005 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16db7e-cedd-44a6-9d41-75783c8b3f81 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:05.658479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16db7e-cedd-44a6-9d41-75783c8b3f81 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:05.658781 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16db7e-cedd-44a6-9d41-75783c8b3f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:05.679569 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4b16db7e-cedd-44a6-9d41-75783c8b3f81 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:05.680920 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4b16db7e-cedd-44a6-9d41-75783c8b3f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:05.743929 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1177/2343] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:05.753832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4090e75-0681-4e79-8800-f6e45b02ae12 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:05.756413 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4090e75-0681-4e79-8800-f6e45b02ae12 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:05.756413 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4090e75-0681-4e79-8800-f6e45b02ae12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:05.775387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-b4090e75-0681-4e79-8800-f6e45b02ae12 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:05.776634 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b4090e75-0681-4e79-8800-f6e45b02ae12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:05.786544 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1167/2344] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:05.792659 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b16db7e-cedd-44a6-9d41-75783c8b3f81 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:05.806245 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b16db7e-cedd-44a6-9d41-75783c8b3f81 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:05.813008 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d27b9-dabd-4298-9161-5b39d296fec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:05.814285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d27b9-dabd-4298-9161-5b39d296fec2 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:05.814796 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d27b9-dabd-4298-9161-5b39d296fec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:05.815247 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d27b9-dabd-4298-9161-5b39d296fec2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:05.816084 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2d27b9-dabd-4298-9161-5b39d296fec2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:48:05.816555 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2d27b9-dabd-4298-9161-5b39d296fec2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:05.818336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2d27b9-dabd-4298-9161-5b39d296fec2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:05.830945 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1178/2345] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:48:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:48:05.835592 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4090e75-0681-4e79-8800-f6e45b02ae12 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:05.846435 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4090e75-0681-4e79-8800-f6e45b02ae12 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:05.851731 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059987fc-eab8-4dd5-9c13-2cdb9d373b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:05.852796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059987fc-eab8-4dd5-9c13-2cdb9d373b7a admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:05.853107 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059987fc-eab8-4dd5-9c13-2cdb9d373b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:05.853658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059987fc-eab8-4dd5-9c13-2cdb9d373b7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:05.854303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059987fc-eab8-4dd5-9c13-2cdb9d373b7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:05.857043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059987fc-eab8-4dd5-9c13-2cdb9d373b7a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:05.869214 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:05.950307 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1168/2346] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:48:05 2022] POST /identity/v3/projects => generated 369 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:48:05.952716 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d27b9-dabd-4298-9161-5b39d296fec2 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:05.965164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d27b9-dabd-4298-9161-5b39d296fec2 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:06.025325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e51d5-e7c3-4d29-9591-12cd5486b34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:06.025963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e51d5-e7c3-4d29-9591-12cd5486b34d admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:06.026244 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e51d5-e7c3-4d29-9591-12cd5486b34d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:06.026516 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e51d5-e7c3-4d29-9591-12cd5486b34d admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:06.027150 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5e51d5-e7c3-4d29-9591-12cd5486b34d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:06.029124 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5e51d5-e7c3-4d29-9591-12cd5486b34d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:06.107202 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1179/2347] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:48:05 2022] POST /identity/v3/users => generated 285 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:48:06.113733 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059987fc-eab8-4dd5-9c13-2cdb9d373b7a admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:06.124348 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059987fc-eab8-4dd5-9c13-2cdb9d373b7a admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:06.129527 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e688b0-0220-48ce-a456-cf42dbefc043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:06.130113 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e688b0-0220-48ce-a456-cf42dbefc043 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:06.130389 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e688b0-0220-48ce-a456-cf42dbefc043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:06.130652 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e688b0-0220-48ce-a456-cf42dbefc043 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:06.131282 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e688b0-0220-48ce-a456-cf42dbefc043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:06.132845 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e688b0-0220-48ce-a456-cf42dbefc043 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:06.142721 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1169/2348] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:06.146890 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e51d5-e7c3-4d29-9591-12cd5486b34d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:06.156955 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e51d5-e7c3-4d29-9591-12cd5486b34d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:06.161664 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eab2b4-9772-47a1-b90c-7cc10b028316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:06.162263 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eab2b4-9772-47a1-b90c-7cc10b028316 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:06.162540 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eab2b4-9772-47a1-b90c-7cc10b028316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:06.162813 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eab2b4-9772-47a1-b90c-7cc10b028316 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:06.163530 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eab2b4-9772-47a1-b90c-7cc10b028316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:06.165120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eab2b4-9772-47a1-b90c-7cc10b028316 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:06.167793 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:06.176973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1180/2349] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:06.181241 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e688b0-0220-48ce-a456-cf42dbefc043 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:06.192059 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e688b0-0220-48ce-a456-cf42dbefc043 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:06.196727 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc71b1ca-5682-4088-a11a-e6cb4465d735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:06.197396 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc71b1ca-5682-4088-a11a-e6cb4465d735 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:06.197679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc71b1ca-5682-4088-a11a-e6cb4465d735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:06.197956 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc71b1ca-5682-4088-a11a-e6cb4465d735 admin admin] PATH_INFO: `/v3/projects/9bb947188f3141b882bb4270acebe0b2/users/8f7ffe74254e4dbd8a5674d813fc6add/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:06.212730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc71b1ca-5682-4088-a11a-e6cb4465d735 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bb947188f3141b882bb4270acebe0b2, user_id=8f7ffe74254e4dbd8a5674d813fc6add, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:06.215475 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc71b1ca-5682-4088-a11a-e6cb4465d735 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:06.301402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1170/2350] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:48:06 2022] PUT /identity/v3/projects/9bb947188f3141b882bb4270acebe0b2/users/8f7ffe74254e4dbd8a5674d813fc6add/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:06.315332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f5523-df56-425c-9c13-200e3d17ec0f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:06.315661 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f5523-df56-425c-9c13-200e3d17ec0f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:06.315940 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f5523-df56-425c-9c13-200e3d17ec0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:06.343279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-f48f5523-df56-425c-9c13-200e3d17ec0f None None] MFA Rules not processed for user `8f7ffe74254e4dbd8a5674d813fc6add`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:06.345575 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f48f5523-df56-425c-9c13-200e3d17ec0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:06.418157 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1181/2351] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:06.423392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc71b1ca-5682-4088-a11a-e6cb4465d735 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:06.490696 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc71b1ca-5682-4088-a11a-e6cb4465d735 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:06.495481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1eeb8-0e92-4ac1-b6cc-cd179652e2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:06.496090 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1eeb8-0e92-4ac1-b6cc-cd179652e2cc admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:06.496369 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1eeb8-0e92-4ac1-b6cc-cd179652e2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:06.496704 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1eeb8-0e92-4ac1-b6cc-cd179652e2cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:06.497288 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e1eeb8-0e92-4ac1-b6cc-cd179652e2cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:06.499178 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e1eeb8-0e92-4ac1-b6cc-cd179652e2cc admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:06.600002 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1171/2352] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:48:06 2022] POST /identity/v3/projects => generated 367 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:48:06.604189 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f5523-df56-425c-9c13-200e3d17ec0f None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:06.615233 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f5523-df56-425c-9c13-200e3d17ec0f None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:06.669392 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c672ac-12f0-4d25-bc36-344b806e316d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:06.670065 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c672ac-12f0-4d25-bc36-344b806e316d admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:06.670256 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c672ac-12f0-4d25-bc36-344b806e316d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:06.670524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c672ac-12f0-4d25-bc36-344b806e316d admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:06.671150 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c672ac-12f0-4d25-bc36-344b806e316d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:06.673048 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c672ac-12f0-4d25-bc36-344b806e316d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:06.745914 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1182/2353] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:48:06 2022] POST /identity/v3/users => generated 290 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:48:06.749892 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1eeb8-0e92-4ac1-b6cc-cd179652e2cc admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:06.760299 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1eeb8-0e92-4ac1-b6cc-cd179652e2cc admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:06.764845 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0065381-de21-48bb-8ac8-fa9d2a25b0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:06.765471 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0065381-de21-48bb-8ac8-fa9d2a25b0a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:06.765749 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0065381-de21-48bb-8ac8-fa9d2a25b0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:06.766018 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0065381-de21-48bb-8ac8-fa9d2a25b0a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:06.766652 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0065381-de21-48bb-8ac8-fa9d2a25b0a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:06.768420 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0065381-de21-48bb-8ac8-fa9d2a25b0a0 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:06.779034 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1172/2354] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:06.783967 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c672ac-12f0-4d25-bc36-344b806e316d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:06.793988 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c672ac-12f0-4d25-bc36-344b806e316d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:06.799134 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff4668-78f6-4e79-89ed-3a69a26f8998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:06.799779 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff4668-78f6-4e79-89ed-3a69a26f8998 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:06.800060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff4668-78f6-4e79-89ed-3a69a26f8998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:06.800336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff4668-78f6-4e79-89ed-3a69a26f8998 admin admin] PATH_INFO: `/v3/projects/830381461ff94d55a3fbee84ff2e5c27/users/a3c8c24eb83743f5ad8aebc590c27ce1/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:06.814897 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff4668-78f6-4e79-89ed-3a69a26f8998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=830381461ff94d55a3fbee84ff2e5c27, user_id=a3c8c24eb83743f5ad8aebc590c27ce1, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:06.817542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff4668-78f6-4e79-89ed-3a69a26f8998 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:06.917605 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1183/2355] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:48:06 2022] PUT /identity/v3/projects/830381461ff94d55a3fbee84ff2e5c27/users/a3c8c24eb83743f5ad8aebc590c27ce1/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:06.921610 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0065381-de21-48bb-8ac8-fa9d2a25b0a0 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:06.932403 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0065381-de21-48bb-8ac8-fa9d2a25b0a0 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:06.988594 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fef94d-657b-4b2d-8d81-d3830ea4d355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:06.989225 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fef94d-657b-4b2d-8d81-d3830ea4d355 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:06.989506 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fef94d-657b-4b2d-8d81-d3830ea4d355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:06.989774 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fef94d-657b-4b2d-8d81-d3830ea4d355 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:06.990409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fef94d-657b-4b2d-8d81-d3830ea4d355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:06.992069 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fef94d-657b-4b2d-8d81-d3830ea4d355 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:07.001950 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1173/2356] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:48:07.006291 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff4668-78f6-4e79-89ed-3a69a26f8998 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:07.016594 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff4668-78f6-4e79-89ed-3a69a26f8998 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:07.021978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf45e2-11e3-4234-a9a0-6aa5d541eb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:07.022625 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf45e2-11e3-4234-a9a0-6aa5d541eb9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:07.022902 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf45e2-11e3-4234-a9a0-6aa5d541eb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:07.023176 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf45e2-11e3-4234-a9a0-6aa5d541eb9d admin admin] PATH_INFO: `/v3/projects/830381461ff94d55a3fbee84ff2e5c27/users/a3c8c24eb83743f5ad8aebc590c27ce1/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:07.025117 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf45e2-11e3-4234-a9a0-6aa5d541eb9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=830381461ff94d55a3fbee84ff2e5c27, user_id=a3c8c24eb83743f5ad8aebc590c27ce1, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:07.027952 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf45e2-11e3-4234-a9a0-6aa5d541eb9d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:07.039211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1184/2357] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:48:07 2022] PUT /identity/v3/projects/830381461ff94d55a3fbee84ff2e5c27/users/a3c8c24eb83743f5ad8aebc590c27ce1/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:07.049178 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaec26a-e64b-4780-b19a-b959c1d696ec None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:07.049739 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaec26a-e64b-4780-b19a-b959c1d696ec None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:07.050021 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaec26a-e64b-4780-b19a-b959c1d696ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:07.077085 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-eeaec26a-e64b-4780-b19a-b959c1d696ec None None] MFA Rules not processed for user `a3c8c24eb83743f5ad8aebc590c27ce1`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:07.078256 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-eeaec26a-e64b-4780-b19a-b959c1d696ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:07.091738 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:07.147092 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1174/2358] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:07 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:07.158460 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf45e2-11e3-4234-a9a0-6aa5d541eb9d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:07.160843 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2eaf45e2-11e3-4234-a9a0-6aa5d541eb9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:07.204766 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:07.220230 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf45e2-11e3-4234-a9a0-6aa5d541eb9d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:07.224659 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aff44c-da5f-45a0-bbbb-6c6d237ac278 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:07.225301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aff44c-da5f-45a0-bbbb-6c6d237ac278 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:07.225585 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aff44c-da5f-45a0-bbbb-6c6d237ac278 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:07.225856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aff44c-da5f-45a0-bbbb-6c6d237ac278 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:07.235139 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aff44c-da5f-45a0-bbbb-6c6d237ac278 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:07.236694 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aff44c-da5f-45a0-bbbb-6c6d237ac278 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:07.246557 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1185/2359] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:48:07 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:48:07.350378 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaec26a-e64b-4780-b19a-b959c1d696ec None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:07.351746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-eeaec26a-e64b-4780-b19a-b959c1d696ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:07.364103 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaec26a-e64b-4780-b19a-b959c1d696ec None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:07.367711 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84b3f7-5186-4d63-9dff-05b039bcc7cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:07.368307 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84b3f7-5186-4d63-9dff-05b039bcc7cf service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:07.368584 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84b3f7-5186-4d63-9dff-05b039bcc7cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:07.368856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84b3f7-5186-4d63-9dff-05b039bcc7cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:07.424365 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84b3f7-5186-4d63-9dff-05b039bcc7cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:07.425835 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84b3f7-5186-4d63-9dff-05b039bcc7cf service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:07.434629 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1175/2360] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:48:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:48:12.218510 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aff44c-da5f-45a0-bbbb-6c6d237ac278 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:12.285219 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aff44c-da5f-45a0-bbbb-6c6d237ac278 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:12.289832 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031bb7f-c7c4-4811-b7a4-99da748e40a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:12.290407 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031bb7f-c7c4-4811-b7a4-99da748e40a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:12.290688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031bb7f-c7c4-4811-b7a4-99da748e40a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:12.290964 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031bb7f-c7c4-4811-b7a4-99da748e40a2 admin admin] PATH_INFO: `/v3/users/8f7ffe74254e4dbd8a5674d813fc6add` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:12.292084 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2031bb7f-c7c4-4811-b7a4-99da748e40a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f7ffe74254e4dbd8a5674d813fc6add)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:12.294032 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2031bb7f-c7c4-4811-b7a4-99da748e40a2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:12.482594 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2031bb7f-c7c4-4811-b7a4-99da748e40a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f7ffe74254e4dbd8a5674d813fc6add'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:12.483245 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2031bb7f-c7c4-4811-b7a4-99da748e40a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f7ffe74254e4dbd8a5674d813fc6add'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:12.483877 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2031bb7f-c7c4-4811-b7a4-99da748e40a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f7ffe74254e4dbd8a5674d813fc6add'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:12.491538 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2031bb7f-c7c4-4811-b7a4-99da748e40a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f7ffe74254e4dbd8a5674d813fc6add'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:12.541183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2031bb7f-c7c4-4811-b7a4-99da748e40a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f7ffe74254e4dbd8a5674d813fc6add'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:12.566970 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1186/2361] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:48:12 2022] DELETE /identity/v3/users/8f7ffe74254e4dbd8a5674d813fc6add => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:12.571607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc0749-0354-4107-b410-318590989132 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:12.572076 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc0749-0354-4107-b410-318590989132 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:12.572182 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc0749-0354-4107-b410-318590989132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:12.593314 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-d4fc0749-0354-4107-b410-318590989132 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:12.594596 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d4fc0749-0354-4107-b410-318590989132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:12.661340 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1176/2362] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:12.671045 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031bb7f-c7c4-4811-b7a4-99da748e40a2 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:12.672472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2031bb7f-c7c4-4811-b7a4-99da748e40a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:12.737460 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031bb7f-c7c4-4811-b7a4-99da748e40a2 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:12.741385 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd53d8-5a4f-45c3-941d-8fd70376aff0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:12.742011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd53d8-5a4f-45c3-941d-8fd70376aff0 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:12.742294 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd53d8-5a4f-45c3-941d-8fd70376aff0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:12.742567 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd53d8-5a4f-45c3-941d-8fd70376aff0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:12.752787 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd53d8-5a4f-45c3-941d-8fd70376aff0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:12.754647 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd53d8-5a4f-45c3-941d-8fd70376aff0 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:12.765360 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1187/2363] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:48:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:48:13.205341 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc0749-0354-4107-b410-318590989132 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:13.206992 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d4fc0749-0354-4107-b410-318590989132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:13.219110 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc0749-0354-4107-b410-318590989132 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:13.224479 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8848d6-0d8f-4b0f-b84c-4d0883d65e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:13.225111 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8848d6-0d8f-4b0f-b84c-4d0883d65e80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:13.225421 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8848d6-0d8f-4b0f-b84c-4d0883d65e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:13.225694 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8848d6-0d8f-4b0f-b84c-4d0883d65e80 admin admin] PATH_INFO: `/v3/projects/9bb947188f3141b882bb4270acebe0b2` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:13.226801 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8848d6-0d8f-4b0f-b84c-4d0883d65e80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bb947188f3141b882bb4270acebe0b2)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:13.228961 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8848d6-0d8f-4b0f-b84c-4d0883d65e80 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:13.357056 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:13.359888 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-6c8848d6-0d8f-4b0f-b84c-4d0883d65e80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bb947188f3141b882bb4270acebe0b2', 'request_id': 'req-6c8848d6-0d8f-4b0f-b84c-4d0883d65e80'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:13.366814 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-6c8848d6-0d8f-4b0f-b84c-4d0883d65e80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bb947188f3141b882bb4270acebe0b2', 'request_id': 'req-6c8848d6-0d8f-4b0f-b84c-4d0883d65e80'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:13.372215 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-6c8848d6-0d8f-4b0f-b84c-4d0883d65e80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bb947188f3141b882bb4270acebe0b2', 'request_id': 'req-6c8848d6-0d8f-4b0f-b84c-4d0883d65e80'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:13.432788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-6c8848d6-0d8f-4b0f-b84c-4d0883d65e80 admin admin] The token cache is being invalidate because project 9bb947188f3141b882bb4270acebe0b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:48:13.433224 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-6c8848d6-0d8f-4b0f-b84c-4d0883d65e80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:13.433855 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-6c8848d6-0d8f-4b0f-b84c-4d0883d65e80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:13.435368 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1177/2364] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:48:13 2022] DELETE /identity/v3/projects/9bb947188f3141b882bb4270acebe0b2 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:13.439488 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd53d8-5a4f-45c3-941d-8fd70376aff0 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:13.445755 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:13.451197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd53d8-5a4f-45c3-941d-8fd70376aff0 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:13.510771 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737f217-922f-4b1d-98c8-84a4047d2812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:13.511433 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737f217-922f-4b1d-98c8-84a4047d2812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:13.511709 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737f217-922f-4b1d-98c8-84a4047d2812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:13.511978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737f217-922f-4b1d-98c8-84a4047d2812 admin admin] PATH_INFO: `/v3/users/a3c8c24eb83743f5ad8aebc590c27ce1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:13.513130 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0737f217-922f-4b1d-98c8-84a4047d2812 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3c8c24eb83743f5ad8aebc590c27ce1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:13.515242 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0737f217-922f-4b1d-98c8-84a4047d2812 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:13.732045 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0737f217-922f-4b1d-98c8-84a4047d2812 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3c8c24eb83743f5ad8aebc590c27ce1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:13.732618 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0737f217-922f-4b1d-98c8-84a4047d2812 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3c8c24eb83743f5ad8aebc590c27ce1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:13.733253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0737f217-922f-4b1d-98c8-84a4047d2812 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3c8c24eb83743f5ad8aebc590c27ce1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:13.740290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0737f217-922f-4b1d-98c8-84a4047d2812 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3c8c24eb83743f5ad8aebc590c27ce1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:13.797882 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0737f217-922f-4b1d-98c8-84a4047d2812 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3c8c24eb83743f5ad8aebc590c27ce1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:13.816444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1188/2365] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:48:13 2022] DELETE /identity/v3/users/a3c8c24eb83743f5ad8aebc590c27ce1 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:13.901212 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea8343-33df-41a2-a138-fa1d0e4371fb None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:13.901726 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea8343-33df-41a2-a138-fa1d0e4371fb None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:13.901852 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea8343-33df-41a2-a138-fa1d0e4371fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:13.923451 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-97ea8343-33df-41a2-a138-fa1d0e4371fb None None] MFA Rules not processed for user `a02c4fdcb5ad418d93e0629f604babea`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:13.924658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-97ea8343-33df-41a2-a138-fa1d0e4371fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:13.993264 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1178/2366] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:14.299081 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737f217-922f-4b1d-98c8-84a4047d2812 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:14.301117 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0737f217-922f-4b1d-98c8-84a4047d2812 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:14.372340 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737f217-922f-4b1d-98c8-84a4047d2812 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:14.377097 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6145a6e9-1be6-429b-852c-1c42eb582c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:14.377762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145a6e9-1be6-429b-852c-1c42eb582c12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:14.378038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145a6e9-1be6-429b-852c-1c42eb582c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:14.379018 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145a6e9-1be6-429b-852c-1c42eb582c12 admin admin] PATH_INFO: `/v3/projects/830381461ff94d55a3fbee84ff2e5c27` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:14.380149 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6145a6e9-1be6-429b-852c-1c42eb582c12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=830381461ff94d55a3fbee84ff2e5c27)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:14.382087 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6145a6e9-1be6-429b-852c-1c42eb582c12 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:14.407153 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:14.529015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6145a6e9-1be6-429b-852c-1c42eb582c12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '830381461ff94d55a3fbee84ff2e5c27', 'request_id': 'req-6145a6e9-1be6-429b-852c-1c42eb582c12'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:14.609195 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6145a6e9-1be6-429b-852c-1c42eb582c12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '830381461ff94d55a3fbee84ff2e5c27', 'request_id': 'req-6145a6e9-1be6-429b-852c-1c42eb582c12'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:14.616364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6145a6e9-1be6-429b-852c-1c42eb582c12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '830381461ff94d55a3fbee84ff2e5c27', 'request_id': 'req-6145a6e9-1be6-429b-852c-1c42eb582c12'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:14.626462 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6145a6e9-1be6-429b-852c-1c42eb582c12 admin admin] The token cache is being invalidate because project 830381461ff94d55a3fbee84ff2e5c27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:48:14.626749 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6145a6e9-1be6-429b-852c-1c42eb582c12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:14.627387 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-6145a6e9-1be6-429b-852c-1c42eb582c12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:14.629544 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1189/2367] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:48:14 2022] DELETE /identity/v3/projects/830381461ff94d55a3fbee84ff2e5c27 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:14.634535 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5d142-dd69-4d50-99b0-868c63eaa13a None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:14.634818 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5d142-dd69-4d50-99b0-868c63eaa13a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:14.635090 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5d142-dd69-4d50-99b0-868c63eaa13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:14.654145 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-bfa5d142-dd69-4d50-99b0-868c63eaa13a None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:14.655322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-bfa5d142-dd69-4d50-99b0-868c63eaa13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:14.719428 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1179/2368] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:14.729079 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4057c14a-9878-4c9f-8916-0e41f1210447 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:14.729518 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4057c14a-9878-4c9f-8916-0e41f1210447 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:14.729895 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4057c14a-9878-4c9f-8916-0e41f1210447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:14.751986 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4057c14a-9878-4c9f-8916-0e41f1210447 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:14.753229 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4057c14a-9878-4c9f-8916-0e41f1210447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:14.761888 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1190/2369] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:14.766323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5d142-dd69-4d50-99b0-868c63eaa13a None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:14.782233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5d142-dd69-4d50-99b0-868c63eaa13a None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:14.789436 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5aa076-3312-4b5c-a63d-63a9743675d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:14.790150 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5aa076-3312-4b5c-a63d-63a9743675d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:14.790551 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5aa076-3312-4b5c-a63d-63a9743675d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:14.790830 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5aa076-3312-4b5c-a63d-63a9743675d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:14.791507 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5aa076-3312-4b5c-a63d-63a9743675d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:48:14.791937 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5aa076-3312-4b5c-a63d-63a9743675d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:14.793561 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5aa076-3312-4b5c-a63d-63a9743675d0 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:14.810263 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1180/2370] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:48:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:48:14.811372 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4057c14a-9878-4c9f-8916-0e41f1210447 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:14.821536 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4057c14a-9878-4c9f-8916-0e41f1210447 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:14.832181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f346551-d3f6-4489-849b-b2078d745e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:14.832181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f346551-d3f6-4489-849b-b2078d745e4a admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:14.832181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f346551-d3f6-4489-849b-b2078d745e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:14.832181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f346551-d3f6-4489-849b-b2078d745e4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:14.832597 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f346551-d3f6-4489-849b-b2078d745e4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:14.832597 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f346551-d3f6-4489-849b-b2078d745e4a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:14.931071 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1191/2371] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:48:14 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:48:14.935525 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5aa076-3312-4b5c-a63d-63a9743675d0 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:14.945944 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5aa076-3312-4b5c-a63d-63a9743675d0 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:15.001138 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b66ad2-e1db-479c-95dd-c52e2114b5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:15.001766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b66ad2-e1db-479c-95dd-c52e2114b5fb admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:15.002112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b66ad2-e1db-479c-95dd-c52e2114b5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:15.002311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b66ad2-e1db-479c-95dd-c52e2114b5fb admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:15.002932 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b66ad2-e1db-479c-95dd-c52e2114b5fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:15.005579 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b66ad2-e1db-479c-95dd-c52e2114b5fb admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:15.016329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:15.123910 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1181/2372] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:48:14 2022] POST /identity/v3/users => generated 292 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:48:15.128126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f346551-d3f6-4489-849b-b2078d745e4a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:15.139474 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f346551-d3f6-4489-849b-b2078d745e4a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:15.144310 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0dd42-413b-4aea-a668-b6af0dc06c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:15.145019 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0dd42-413b-4aea-a668-b6af0dc06c63 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:15.145260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0dd42-413b-4aea-a668-b6af0dc06c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:15.145533 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0dd42-413b-4aea-a668-b6af0dc06c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:15.146170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0dd42-413b-4aea-a668-b6af0dc06c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:15.147804 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0dd42-413b-4aea-a668-b6af0dc06c63 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:15.157685 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1192/2373] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:15.161914 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b66ad2-e1db-479c-95dd-c52e2114b5fb admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:15.171926 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b66ad2-e1db-479c-95dd-c52e2114b5fb admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:15.177392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda129c-c70e-4b3f-a45d-7512570d17a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:15.177983 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda129c-c70e-4b3f-a45d-7512570d17a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:15.178259 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda129c-c70e-4b3f-a45d-7512570d17a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:15.178528 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda129c-c70e-4b3f-a45d-7512570d17a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:15.179392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda129c-c70e-4b3f-a45d-7512570d17a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:15.181009 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda129c-c70e-4b3f-a45d-7512570d17a7 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:15.192578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1182/2374] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:15.197056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0dd42-413b-4aea-a668-b6af0dc06c63 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:15.207316 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0dd42-413b-4aea-a668-b6af0dc06c63 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:15.211816 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e44db0-a720-4321-a6b9-46bdb33ba9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:15.212454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e44db0-a720-4321-a6b9-46bdb33ba9be admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:15.212733 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e44db0-a720-4321-a6b9-46bdb33ba9be admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:15.213005 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e44db0-a720-4321-a6b9-46bdb33ba9be admin admin] PATH_INFO: `/v3/projects/ea9f95038e5d4ac3aa33ebdce5f63055/users/d298b4324a304b8e9d1deeac7420b83d/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:15.227254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e44db0-a720-4321-a6b9-46bdb33ba9be admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea9f95038e5d4ac3aa33ebdce5f63055, user_id=d298b4324a304b8e9d1deeac7420b83d, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:15.230219 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e44db0-a720-4321-a6b9-46bdb33ba9be admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:15.330098 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1193/2375] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:48:15 2022] PUT /identity/v3/projects/ea9f95038e5d4ac3aa33ebdce5f63055/users/d298b4324a304b8e9d1deeac7420b83d/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:15.342915 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125ce08-5ff3-4e7b-a915-2d6911c35190 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:15.343206 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125ce08-5ff3-4e7b-a915-2d6911c35190 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:15.343484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125ce08-5ff3-4e7b-a915-2d6911c35190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:15.379144 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-8125ce08-5ff3-4e7b-a915-2d6911c35190 None None] MFA Rules not processed for user `d298b4324a304b8e9d1deeac7420b83d`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:15.380357 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8125ce08-5ff3-4e7b-a915-2d6911c35190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:15.401281 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e44db0-a720-4321-a6b9-46bdb33ba9be admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:15.403188 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-25e44db0-a720-4321-a6b9-46bdb33ba9be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:15.455936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1183/2376] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:15.460523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125ce08-5ff3-4e7b-a915-2d6911c35190 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:15.466903 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e44db0-a720-4321-a6b9-46bdb33ba9be admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:15.470696 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e139f-00be-4c94-876a-5f17f48eea0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:15.471298 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e139f-00be-4c94-876a-5f17f48eea0d service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:15.471574 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e139f-00be-4c94-876a-5f17f48eea0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:15.472367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e139f-00be-4c94-876a-5f17f48eea0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:15.474317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-856e139f-00be-4c94-876a-5f17f48eea0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:15.524721 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125ce08-5ff3-4e7b-a915-2d6911c35190 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:15.529308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57d99a-74bc-49fd-99b0-2c109da16b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:15.529896 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57d99a-74bc-49fd-99b0-2c109da16b7e admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:15.530173 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57d99a-74bc-49fd-99b0-2c109da16b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:15.530443 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57d99a-74bc-49fd-99b0-2c109da16b7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:15.531084 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57d99a-74bc-49fd-99b0-2c109da16b7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:15.533043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57d99a-74bc-49fd-99b0-2c109da16b7e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:15.539411 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856e139f-00be-4c94-876a-5f17f48eea0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:15.539846 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856e139f-00be-4c94-876a-5f17f48eea0d service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:15.543680 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:15.550090 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1194/2377] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:48:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:48:15.639509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1184/2378] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:48:15 2022] POST /identity/v3/projects => generated 385 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:48:15.643566 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e139f-00be-4c94-876a-5f17f48eea0d service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:15.654583 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e139f-00be-4c94-876a-5f17f48eea0d service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:15.711724 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b05b1-5f81-43f7-9723-26b22f11b7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:15.712336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b05b1-5f81-43f7-9723-26b22f11b7a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:15.712617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b05b1-5f81-43f7-9723-26b22f11b7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:15.712891 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b05b1-5f81-43f7-9723-26b22f11b7a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:15.713554 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558b05b1-5f81-43f7-9723-26b22f11b7a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:15.715495 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558b05b1-5f81-43f7-9723-26b22f11b7a2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:15.811340 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1195/2379] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:48:15 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:48:15.815288 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57d99a-74bc-49fd-99b0-2c109da16b7e admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:15.827235 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57d99a-74bc-49fd-99b0-2c109da16b7e admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:15.833862 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146575c1-203f-40e2-b8d0-709bf0ddc3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:15.834564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146575c1-203f-40e2-b8d0-709bf0ddc3ee admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:15.835036 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146575c1-203f-40e2-b8d0-709bf0ddc3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:15.835382 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146575c1-203f-40e2-b8d0-709bf0ddc3ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:15.836138 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146575c1-203f-40e2-b8d0-709bf0ddc3ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:15.838199 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146575c1-203f-40e2-b8d0-709bf0ddc3ee admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:15.859826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1185/2380] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:15.864591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b05b1-5f81-43f7-9723-26b22f11b7a2 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:15.876869 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b05b1-5f81-43f7-9723-26b22f11b7a2 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:15.895783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e45499-c84d-4363-9e66-ce8a1959614d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:15.896531 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e45499-c84d-4363-9e66-ce8a1959614d admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:15.896909 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e45499-c84d-4363-9e66-ce8a1959614d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:15.897344 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e45499-c84d-4363-9e66-ce8a1959614d admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:15.898115 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e45499-c84d-4363-9e66-ce8a1959614d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:15.899962 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e45499-c84d-4363-9e66-ce8a1959614d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:15.914951 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1196/2381] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:15.916212 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146575c1-203f-40e2-b8d0-709bf0ddc3ee admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:15.928193 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146575c1-203f-40e2-b8d0-709bf0ddc3ee admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:15.933052 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5120019-f156-4406-9c15-e625ae85c4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:15.933740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5120019-f156-4406-9c15-e625ae85c4ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:15.934052 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5120019-f156-4406-9c15-e625ae85c4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:15.934323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5120019-f156-4406-9c15-e625ae85c4ba admin admin] PATH_INFO: `/v3/projects/1a7f46cdc0314386950ebb7afba1455f/users/ffbe3433c0c14606b5161246ea956473/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:15.950586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5120019-f156-4406-9c15-e625ae85c4ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a7f46cdc0314386950ebb7afba1455f, user_id=ffbe3433c0c14606b5161246ea956473, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:15.953278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5120019-f156-4406-9c15-e625ae85c4ba admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:16.043712 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1186/2382] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:48:15 2022] PUT /identity/v3/projects/1a7f46cdc0314386950ebb7afba1455f/users/ffbe3433c0c14606b5161246ea956473/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:48:16.055499 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23be53b-91e5-4bb1-abe2-a658b2b6222f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:16.055827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23be53b-91e5-4bb1-abe2-a658b2b6222f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:16.056232 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23be53b-91e5-4bb1-abe2-a658b2b6222f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:16.088769 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a23be53b-91e5-4bb1-abe2-a658b2b6222f None None] MFA Rules not processed for user `ffbe3433c0c14606b5161246ea956473`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:16.090190 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a23be53b-91e5-4bb1-abe2-a658b2b6222f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:16.160413 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1197/2383] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:16.164945 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5120019-f156-4406-9c15-e625ae85c4ba admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:16.214308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:16.250038 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5120019-f156-4406-9c15-e625ae85c4ba admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:16.255399 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ddd6b-589f-4955-836a-e2f1b6d99cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:16.255997 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ddd6b-589f-4955-836a-e2f1b6d99cfa admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:16.256272 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ddd6b-589f-4955-836a-e2f1b6d99cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:16.256540 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ddd6b-589f-4955-836a-e2f1b6d99cfa admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:16.257192 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ddd6b-589f-4955-836a-e2f1b6d99cfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:16.259073 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ddd6b-589f-4955-836a-e2f1b6d99cfa admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:16.374802 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1187/2384] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:48:16 2022] POST /identity/v3/projects => generated 387 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:48:16.375976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23be53b-91e5-4bb1-abe2-a658b2b6222f None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:16.389513 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23be53b-91e5-4bb1-abe2-a658b2b6222f None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:16.455687 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874594e-bd9f-4ec4-b45b-7b191be143cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:16.456653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874594e-bd9f-4ec4-b45b-7b191be143cb admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:16.457013 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874594e-bd9f-4ec4-b45b-7b191be143cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:16.457374 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874594e-bd9f-4ec4-b45b-7b191be143cb admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:16.458067 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874594e-bd9f-4ec4-b45b-7b191be143cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:16.460470 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874594e-bd9f-4ec4-b45b-7b191be143cb admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:16.471565 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:16.570593 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1198/2385] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:48:16 2022] POST /identity/v3/users => generated 300 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:48:16.574376 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ddd6b-589f-4955-836a-e2f1b6d99cfa admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:16.585752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ddd6b-589f-4955-836a-e2f1b6d99cfa admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:16.590809 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2535c-adca-4ef2-9beb-68fac75425d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:16.591466 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2535c-adca-4ef2-9beb-68fac75425d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:16.591770 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2535c-adca-4ef2-9beb-68fac75425d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:16.592043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2535c-adca-4ef2-9beb-68fac75425d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:16.592692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2535c-adca-4ef2-9beb-68fac75425d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:16.594931 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2535c-adca-4ef2-9beb-68fac75425d4 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:16.607893 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1188/2386] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:16.614851 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874594e-bd9f-4ec4-b45b-7b191be143cb admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:16.626387 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874594e-bd9f-4ec4-b45b-7b191be143cb admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:16.631492 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f017f78-163f-410c-b68a-68a6581b4e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:16.632160 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f017f78-163f-410c-b68a-68a6581b4e22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:16.632435 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f017f78-163f-410c-b68a-68a6581b4e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:16.632735 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f017f78-163f-410c-b68a-68a6581b4e22 admin admin] PATH_INFO: `/v3/projects/e83a1cb21e1d4d2880d7ffce97247a96/users/0f146715642a482f837d4b46d320b751/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:16.650577 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f017f78-163f-410c-b68a-68a6581b4e22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e83a1cb21e1d4d2880d7ffce97247a96, user_id=0f146715642a482f837d4b46d320b751, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:16.656040 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f017f78-163f-410c-b68a-68a6581b4e22 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:16.747862 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1199/2387] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:48:16 2022] PUT /identity/v3/projects/e83a1cb21e1d4d2880d7ffce97247a96/users/0f146715642a482f837d4b46d320b751/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:16.752669 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2535c-adca-4ef2-9beb-68fac75425d4 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:16.764014 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2535c-adca-4ef2-9beb-68fac75425d4 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:16.838993 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43016e5c-5f88-44e3-a24e-d71229db2c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:16.839753 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43016e5c-5f88-44e3-a24e-d71229db2c0a admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:16.840044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43016e5c-5f88-44e3-a24e-d71229db2c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:16.840403 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43016e5c-5f88-44e3-a24e-d71229db2c0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:16.841059 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43016e5c-5f88-44e3-a24e-d71229db2c0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:16.844351 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43016e5c-5f88-44e3-a24e-d71229db2c0a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:16.862243 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1189/2388] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:16.865643 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f017f78-163f-410c-b68a-68a6581b4e22 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:16.893195 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f017f78-163f-410c-b68a-68a6581b4e22 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:16.901091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d4c21-7797-4e93-90f8-f545f60ae113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:16.901878 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d4c21-7797-4e93-90f8-f545f60ae113 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:16.902228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d4c21-7797-4e93-90f8-f545f60ae113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:16.902568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d4c21-7797-4e93-90f8-f545f60ae113 admin admin] PATH_INFO: `/v3/projects/e83a1cb21e1d4d2880d7ffce97247a96/users/0f146715642a482f837d4b46d320b751/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:16.905295 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d4c21-7797-4e93-90f8-f545f60ae113 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e83a1cb21e1d4d2880d7ffce97247a96, user_id=0f146715642a482f837d4b46d320b751, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:16.924393 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d4c21-7797-4e93-90f8-f545f60ae113 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:16.924393 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1200/2389] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:48:16 2022] PUT /identity/v3/projects/e83a1cb21e1d4d2880d7ffce97247a96/users/0f146715642a482f837d4b46d320b751/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:48:16.934513 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76575db3-d764-4282-b2bb-1a0c21acee96 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:16.934793 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76575db3-d764-4282-b2bb-1a0c21acee96 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:16.935070 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76575db3-d764-4282-b2bb-1a0c21acee96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:16.969789 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-76575db3-d764-4282-b2bb-1a0c21acee96 None None] MFA Rules not processed for user `0f146715642a482f837d4b46d320b751`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:16.971059 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-76575db3-d764-4282-b2bb-1a0c21acee96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:17.076597 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1190/2390] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:16 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:17.085069 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d4c21-7797-4e93-90f8-f545f60ae113 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:17.087902 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-752d4c21-7797-4e93-90f8-f545f60ae113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:17.224532 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d4c21-7797-4e93-90f8-f545f60ae113 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:17.228832 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158afa3-3a8c-4032-97b8-ca4f5d86fdb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:17.229591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158afa3-3a8c-4032-97b8-ca4f5d86fdb2 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:17.229869 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158afa3-3a8c-4032-97b8-ca4f5d86fdb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:17.230136 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158afa3-3a8c-4032-97b8-ca4f5d86fdb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:17.242510 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158afa3-3a8c-4032-97b8-ca4f5d86fdb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:17.244788 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158afa3-3a8c-4032-97b8-ca4f5d86fdb2 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:17.256916 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1201/2391] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:48:17 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:48:18.621215 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76575db3-d764-4282-b2bb-1a0c21acee96 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:18.622665 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-76575db3-d764-4282-b2bb-1a0c21acee96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:18.637303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76575db3-d764-4282-b2bb-1a0c21acee96 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:18.641185 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01b73f-a1f7-40a1-930f-8abd3918a69b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:18.641824 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01b73f-a1f7-40a1-930f-8abd3918a69b service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:18.642101 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01b73f-a1f7-40a1-930f-8abd3918a69b service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:18.642368 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01b73f-a1f7-40a1-930f-8abd3918a69b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:18.665388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:18.718204 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb01b73f-a1f7-40a1-930f-8abd3918a69b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:18.732309 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb01b73f-a1f7-40a1-930f-8abd3918a69b service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:18.732309 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1191/2392] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:48:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:48:20.038213 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158afa3-3a8c-4032-97b8-ca4f5d86fdb2 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:20.039873 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5158afa3-3a8c-4032-97b8-ca4f5d86fdb2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:20.076850 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01b73f-a1f7-40a1-930f-8abd3918a69b service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:20.078784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-cb01b73f-a1f7-40a1-930f-8abd3918a69b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:20.203818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158afa3-3a8c-4032-97b8-ca4f5d86fdb2 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:20.218727 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f0142-26e1-4c7e-b395-56f3bcd6866d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:20.219294 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f0142-26e1-4c7e-b395-56f3bcd6866d service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:20.219576 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f0142-26e1-4c7e-b395-56f3bcd6866d service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:20.219843 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f0142-26e1-4c7e-b395-56f3bcd6866d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:20.221715 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-015f0142-26e1-4c7e-b395-56f3bcd6866d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:20.224880 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01b73f-a1f7-40a1-930f-8abd3918a69b service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:20.245816 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780894e3-3da0-4256-83ba-d5d98fe6e85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:20.249418 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780894e3-3da0-4256-83ba-d5d98fe6e85f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:20.249418 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780894e3-3da0-4256-83ba-d5d98fe6e85f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:20.249418 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780894e3-3da0-4256-83ba-d5d98fe6e85f admin admin] PATH_INFO: `/v3/users/81d6a9d89e394e63ba5c8182a12b4142` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:20.252239 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780894e3-3da0-4256-83ba-d5d98fe6e85f admin admin] RBAC: Authorizing `identity:delete_user(user_id=81d6a9d89e394e63ba5c8182a12b4142)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:20.255031 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780894e3-3da0-4256-83ba-d5d98fe6e85f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:20.256056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f0142-26e1-4c7e-b395-56f3bcd6866d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:20.259832 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f0142-26e1-4c7e-b395-56f3bcd6866d service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:20.273929 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1202/2393] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:48:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:48:20.653618 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-780894e3-3da0-4256-83ba-d5d98fe6e85f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81d6a9d89e394e63ba5c8182a12b4142'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:20.655471 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-780894e3-3da0-4256-83ba-d5d98fe6e85f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81d6a9d89e394e63ba5c8182a12b4142'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:20.658537 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-780894e3-3da0-4256-83ba-d5d98fe6e85f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81d6a9d89e394e63ba5c8182a12b4142'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:20.670753 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-780894e3-3da0-4256-83ba-d5d98fe6e85f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81d6a9d89e394e63ba5c8182a12b4142'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:20.745984 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-780894e3-3da0-4256-83ba-d5d98fe6e85f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81d6a9d89e394e63ba5c8182a12b4142'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:20.772095 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1192/2394] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:48:20 2022] DELETE /identity/v3/users/81d6a9d89e394e63ba5c8182a12b4142 => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:21.209484 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f0142-26e1-4c7e-b395-56f3bcd6866d service nova] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:21.224210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f0142-26e1-4c7e-b395-56f3bcd6866d service nova] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:21.288264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:21.303048 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a98de2-29a0-4489-ada2-64713111c533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:21.303785 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a98de2-29a0-4489-ada2-64713111c533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:21.304172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a98de2-29a0-4489-ada2-64713111c533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:21.304592 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a98de2-29a0-4489-ada2-64713111c533 admin admin] PATH_INFO: `/v3/projects/734d6a79fd5845f19569b2c2825fb38c` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:21.306145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a98de2-29a0-4489-ada2-64713111c533 admin admin] RBAC: Authorizing `identity:delete_project(project_id=734d6a79fd5845f19569b2c2825fb38c)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:21.308196 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a98de2-29a0-4489-ada2-64713111c533 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:21.447448 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e8a98de2-29a0-4489-ada2-64713111c533 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '734d6a79fd5845f19569b2c2825fb38c', 'request_id': 'req-e8a98de2-29a0-4489-ada2-64713111c533'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:21.574668 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e8a98de2-29a0-4489-ada2-64713111c533 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '734d6a79fd5845f19569b2c2825fb38c', 'request_id': 'req-e8a98de2-29a0-4489-ada2-64713111c533'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:21.587193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e8a98de2-29a0-4489-ada2-64713111c533 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '734d6a79fd5845f19569b2c2825fb38c', 'request_id': 'req-e8a98de2-29a0-4489-ada2-64713111c533'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:21.604821 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e8a98de2-29a0-4489-ada2-64713111c533 admin admin] The token cache is being invalidate because project 734d6a79fd5845f19569b2c2825fb38c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:48:21.608056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e8a98de2-29a0-4489-ada2-64713111c533 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:21.608056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e8a98de2-29a0-4489-ada2-64713111c533 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:21.608056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1203/2395] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:48:21 2022] DELETE /identity/v3/projects/734d6a79fd5845f19569b2c2825fb38c => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:21.613239 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0451c-f45c-414b-b6ab-dcd631d6c84c None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:21.613920 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0451c-f45c-414b-b6ab-dcd631d6c84c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:21.614326 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0451c-f45c-414b-b6ab-dcd631d6c84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:21.641387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-b1f0451c-f45c-414b-b6ab-dcd631d6c84c None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:21.642951 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b1f0451c-f45c-414b-b6ab-dcd631d6c84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:21.678168 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:21.704151 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1193/2396] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:48:21.712440 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c288c5-981b-49cf-8d18-6db5d3602d89 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:21.712726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c288c5-981b-49cf-8d18-6db5d3602d89 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:21.712998 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c288c5-981b-49cf-8d18-6db5d3602d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:21.734119 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-04c288c5-981b-49cf-8d18-6db5d3602d89 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:21.735306 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-04c288c5-981b-49cf-8d18-6db5d3602d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:21.744038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1204/2397] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:21.748321 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0451c-f45c-414b-b6ab-dcd631d6c84c None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:21.758388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0451c-f45c-414b-b6ab-dcd631d6c84c None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:21.763013 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d4bff-5d8f-4590-a291-b447b4611e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:21.763613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d4bff-5d8f-4590-a291-b447b4611e2b admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:21.763897 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d4bff-5d8f-4590-a291-b447b4611e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:21.764169 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d4bff-5d8f-4590-a291-b447b4611e2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:21.764850 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283d4bff-5d8f-4590-a291-b447b4611e2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:48:21.765176 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283d4bff-5d8f-4590-a291-b447b4611e2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:21.766824 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283d4bff-5d8f-4590-a291-b447b4611e2b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:21.779066 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1194/2398] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:48:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:48:21.783505 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c288c5-981b-49cf-8d18-6db5d3602d89 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:21.794339 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c288c5-981b-49cf-8d18-6db5d3602d89 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:21.800464 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c675d6b8-4dcb-41ea-ae64-a9f318015fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:21.802502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675d6b8-4dcb-41ea-ae64-a9f318015fa8 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:21.802502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675d6b8-4dcb-41ea-ae64-a9f318015fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:21.802502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675d6b8-4dcb-41ea-ae64-a9f318015fa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:21.802502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c675d6b8-4dcb-41ea-ae64-a9f318015fa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:21.804573 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c675d6b8-4dcb-41ea-ae64-a9f318015fa8 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:21.913630 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1205/2399] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:48:21 2022] POST /identity/v3/projects => generated 371 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:48:21.917814 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d4bff-5d8f-4590-a291-b447b4611e2b admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:21.928636 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d4bff-5d8f-4590-a291-b447b4611e2b admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:22.026141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a0bc5-a723-4ee9-bb6a-e19a3048f3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:22.027030 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a0bc5-a723-4ee9-bb6a-e19a3048f3eb admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:22.027312 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a0bc5-a723-4ee9-bb6a-e19a3048f3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:22.027586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a0bc5-a723-4ee9-bb6a-e19a3048f3eb admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:22.028211 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a0bc5-a723-4ee9-bb6a-e19a3048f3eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:22.030185 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a0bc5-a723-4ee9-bb6a-e19a3048f3eb admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:22.128562 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c675d6b8-4dcb-41ea-ae64-a9f318015fa8 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:22.138314 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1195/2400] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:48:21 2022] POST /identity/v3/users => generated 286 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:48:22.142783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c675d6b8-4dcb-41ea-ae64-a9f318015fa8 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:22.150215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a153ceb-f458-466d-9caf-a846b341bccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:22.151179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a153ceb-f458-466d-9caf-a846b341bccb admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:22.151572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a153ceb-f458-466d-9caf-a846b341bccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:22.151915 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a153ceb-f458-466d-9caf-a846b341bccb admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:22.152730 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a153ceb-f458-466d-9caf-a846b341bccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:22.155562 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a153ceb-f458-466d-9caf-a846b341bccb admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:22.167792 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1206/2401] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:22.171680 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a0bc5-a723-4ee9-bb6a-e19a3048f3eb admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:22.182702 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a0bc5-a723-4ee9-bb6a-e19a3048f3eb admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:22.188042 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8923e2-5b4f-41b2-aebe-68faa6e3f485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:22.188650 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8923e2-5b4f-41b2-aebe-68faa6e3f485 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:22.188933 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8923e2-5b4f-41b2-aebe-68faa6e3f485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:22.189302 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8923e2-5b4f-41b2-aebe-68faa6e3f485 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:22.189952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8923e2-5b4f-41b2-aebe-68faa6e3f485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:22.192533 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8923e2-5b4f-41b2-aebe-68faa6e3f485 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:22.208935 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1196/2402] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:22.210090 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a153ceb-f458-466d-9caf-a846b341bccb admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:22.223319 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a153ceb-f458-466d-9caf-a846b341bccb admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:22.234047 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d899247-0c1e-4ea3-94d0-0215a3103c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:22.234047 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d899247-0c1e-4ea3-94d0-0215a3103c22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:22.234047 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d899247-0c1e-4ea3-94d0-0215a3103c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:22.234827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d899247-0c1e-4ea3-94d0-0215a3103c22 admin admin] PATH_INFO: `/v3/projects/8c01599e43d749f6945a0917754d5774/users/66b05cea9ff041acaaf255749805ef67/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:22.250884 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d899247-0c1e-4ea3-94d0-0215a3103c22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c01599e43d749f6945a0917754d5774, user_id=66b05cea9ff041acaaf255749805ef67, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:22.253765 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d899247-0c1e-4ea3-94d0-0215a3103c22 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:22.353014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1207/2403] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:48:22 2022] PUT /identity/v3/projects/8c01599e43d749f6945a0917754d5774/users/66b05cea9ff041acaaf255749805ef67/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:22.364918 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fefb51-e7db-4d0e-a913-1397a76ee8ec None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:22.365321 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fefb51-e7db-4d0e-a913-1397a76ee8ec None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:22.365729 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fefb51-e7db-4d0e-a913-1397a76ee8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:22.397118 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-e3fefb51-e7db-4d0e-a913-1397a76ee8ec None None] MFA Rules not processed for user `66b05cea9ff041acaaf255749805ef67`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:22.398358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e3fefb51-e7db-4d0e-a913-1397a76ee8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:22.504735 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1197/2404] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:22.509451 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d899247-0c1e-4ea3-94d0-0215a3103c22 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:22.576728 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d899247-0c1e-4ea3-94d0-0215a3103c22 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:22.582552 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c4dad-114c-4069-a3f3-9a26d588beec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:22.583167 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c4dad-114c-4069-a3f3-9a26d588beec admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:22.583445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c4dad-114c-4069-a3f3-9a26d588beec admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:22.583717 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c4dad-114c-4069-a3f3-9a26d588beec admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:22.584372 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c4dad-114c-4069-a3f3-9a26d588beec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:22.586293 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c4dad-114c-4069-a3f3-9a26d588beec admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:22.672789 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:22.690832 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1208/2405] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:48:22 2022] POST /identity/v3/projects => generated 371 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:48:22.693796 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fefb51-e7db-4d0e-a913-1397a76ee8ec None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:22.704854 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fefb51-e7db-4d0e-a913-1397a76ee8ec None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:22.735432 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:22.760991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15beb409-7f24-4e3b-8020-f57007874f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:22.761637 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15beb409-7f24-4e3b-8020-f57007874f2b admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:22.761919 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15beb409-7f24-4e3b-8020-f57007874f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:22.762191 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15beb409-7f24-4e3b-8020-f57007874f2b admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:22.762806 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15beb409-7f24-4e3b-8020-f57007874f2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:22.764698 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15beb409-7f24-4e3b-8020-f57007874f2b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:22.837814 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1198/2406] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:48:22 2022] POST /identity/v3/users => generated 292 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:48:22.842172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c4dad-114c-4069-a3f3-9a26d588beec admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:22.853253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c4dad-114c-4069-a3f3-9a26d588beec admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:22.858148 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf92458-629c-46f7-8827-1662933c1fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:22.858746 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf92458-629c-46f7-8827-1662933c1fdd admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:22.859025 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf92458-629c-46f7-8827-1662933c1fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:22.859293 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf92458-629c-46f7-8827-1662933c1fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:22.859932 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf92458-629c-46f7-8827-1662933c1fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:22.861630 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf92458-629c-46f7-8827-1662933c1fdd admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:22.873419 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1209/2407] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:22.877778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15beb409-7f24-4e3b-8020-f57007874f2b admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:22.887954 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15beb409-7f24-4e3b-8020-f57007874f2b admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:22.892452 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2b507-902d-4d99-bd47-4e277e64ddc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:22.893086 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2b507-902d-4d99-bd47-4e277e64ddc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:22.893964 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2b507-902d-4d99-bd47-4e277e64ddc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:22.894433 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2b507-902d-4d99-bd47-4e277e64ddc8 admin admin] PATH_INFO: `/v3/projects/2d78e651c1f5403cb7d1de8195176e6e/users/d0a32dc3d81e42288f65468cc6e964b1/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:22.908628 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f2b507-902d-4d99-bd47-4e277e64ddc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d78e651c1f5403cb7d1de8195176e6e, user_id=d0a32dc3d81e42288f65468cc6e964b1, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:22.912032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f2b507-902d-4d99-bd47-4e277e64ddc8 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:22.980317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1199/2408] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:48:22 2022] PUT /identity/v3/projects/2d78e651c1f5403cb7d1de8195176e6e/users/d0a32dc3d81e42288f65468cc6e964b1/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:48:22.984357 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf92458-629c-46f7-8827-1662933c1fdd admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:22.994293 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf92458-629c-46f7-8827-1662933c1fdd admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:23.052634 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c698e-43d2-4852-b7e7-ebac7f7f1a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:23.053381 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c698e-43d2-4852-b7e7-ebac7f7f1a93 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:23.053656 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c698e-43d2-4852-b7e7-ebac7f7f1a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:23.053997 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c698e-43d2-4852-b7e7-ebac7f7f1a93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:23.054686 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c698e-43d2-4852-b7e7-ebac7f7f1a93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:23.056286 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c698e-43d2-4852-b7e7-ebac7f7f1a93 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:23.066657 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1210/2409] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:23.071051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2b507-902d-4d99-bd47-4e277e64ddc8 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:23.081832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2b507-902d-4d99-bd47-4e277e64ddc8 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:23.086471 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6290f-0e91-46eb-9f04-d96dd8d13221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:23.087114 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6290f-0e91-46eb-9f04-d96dd8d13221 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:23.087385 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6290f-0e91-46eb-9f04-d96dd8d13221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:23.087650 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6290f-0e91-46eb-9f04-d96dd8d13221 admin admin] PATH_INFO: `/v3/projects/2d78e651c1f5403cb7d1de8195176e6e/users/d0a32dc3d81e42288f65468cc6e964b1/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:23.089684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6290f-0e91-46eb-9f04-d96dd8d13221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d78e651c1f5403cb7d1de8195176e6e, user_id=d0a32dc3d81e42288f65468cc6e964b1, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:23.094674 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6290f-0e91-46eb-9f04-d96dd8d13221 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:23.102957 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1200/2410] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:48:23 2022] PUT /identity/v3/projects/2d78e651c1f5403cb7d1de8195176e6e/users/d0a32dc3d81e42288f65468cc6e964b1/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:23.114890 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cd938-58c8-42fd-b95f-331c1e81a12c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:23.115182 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cd938-58c8-42fd-b95f-331c1e81a12c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:23.115462 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cd938-58c8-42fd-b95f-331c1e81a12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:23.143145 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-176cd938-58c8-42fd-b95f-331c1e81a12c None None] MFA Rules not processed for user `d0a32dc3d81e42288f65468cc6e964b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:23.145070 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-176cd938-58c8-42fd-b95f-331c1e81a12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:23.214055 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1211/2411] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:23 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:23.225875 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6290f-0e91-46eb-9f04-d96dd8d13221 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:23.227736 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-77f6290f-0e91-46eb-9f04-d96dd8d13221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:23.291669 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6290f-0e91-46eb-9f04-d96dd8d13221 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:23.295380 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183906f9-90e0-46aa-8957-4e546fccf899 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:23.295991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183906f9-90e0-46aa-8957-4e546fccf899 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:23.296269 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183906f9-90e0-46aa-8957-4e546fccf899 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:23.296670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183906f9-90e0-46aa-8957-4e546fccf899 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:23.306742 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183906f9-90e0-46aa-8957-4e546fccf899 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:23.308166 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183906f9-90e0-46aa-8957-4e546fccf899 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:23.318377 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1201/2412] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:48:23 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:48:23.802256 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176cd938-58c8-42fd-b95f-331c1e81a12c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:23.803700 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-176cd938-58c8-42fd-b95f-331c1e81a12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:23.823701 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176cd938-58c8-42fd-b95f-331c1e81a12c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:23.827339 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7549e13-d417-492c-9549-aaff6e689316 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:23.827946 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7549e13-d417-492c-9549-aaff6e689316 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:23.828226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7549e13-d417-492c-9549-aaff6e689316 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:23.828496 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7549e13-d417-492c-9549-aaff6e689316 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:23.830254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e7549e13-d417-492c-9549-aaff6e689316 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:23.897515 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7549e13-d417-492c-9549-aaff6e689316 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:23.898950 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7549e13-d417-492c-9549-aaff6e689316 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:23.907936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1212/2413] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:48:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:48:24.706605 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183906f9-90e0-46aa-8957-4e546fccf899 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:24.771882 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183906f9-90e0-46aa-8957-4e546fccf899 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:24.776463 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb7c1a-c5b3-4121-bba4-0271c4072f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:24.777090 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb7c1a-c5b3-4121-bba4-0271c4072f0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:24.777399 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb7c1a-c5b3-4121-bba4-0271c4072f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:24.777670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb7c1a-c5b3-4121-bba4-0271c4072f0c admin admin] PATH_INFO: `/v3/users/66b05cea9ff041acaaf255749805ef67` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:24.778782 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fb7c1a-c5b3-4121-bba4-0271c4072f0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=66b05cea9ff041acaaf255749805ef67)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:24.780739 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fb7c1a-c5b3-4121-bba4-0271c4072f0c admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:24.998973 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:25.017049 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c6fb7c1a-c5b3-4121-bba4-0271c4072f0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66b05cea9ff041acaaf255749805ef67'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:25.017763 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c6fb7c1a-c5b3-4121-bba4-0271c4072f0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66b05cea9ff041acaaf255749805ef67'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:25.018993 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c6fb7c1a-c5b3-4121-bba4-0271c4072f0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66b05cea9ff041acaaf255749805ef67'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:25.026886 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c6fb7c1a-c5b3-4121-bba4-0271c4072f0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66b05cea9ff041acaaf255749805ef67'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:25.129023 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c6fb7c1a-c5b3-4121-bba4-0271c4072f0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66b05cea9ff041acaaf255749805ef67'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:25.148711 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1202/2414] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:48:24 2022] DELETE /identity/v3/users/66b05cea9ff041acaaf255749805ef67 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:25.153166 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106cac3f-b2c7-4a5d-9855-1f2194ea503c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:25.153701 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106cac3f-b2c7-4a5d-9855-1f2194ea503c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:25.153817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106cac3f-b2c7-4a5d-9855-1f2194ea503c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:25.167671 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:25.173635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-106cac3f-b2c7-4a5d-9855-1f2194ea503c None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:25.174944 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-106cac3f-b2c7-4a5d-9855-1f2194ea503c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:25.235201 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1213/2415] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:25.245181 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb7c1a-c5b3-4121-bba4-0271c4072f0c admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:25.246600 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c6fb7c1a-c5b3-4121-bba4-0271c4072f0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:25.304194 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb7c1a-c5b3-4121-bba4-0271c4072f0c admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:25.307882 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d5774-07b0-4316-a7b1-0b41bc5dc4b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:25.308486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d5774-07b0-4316-a7b1-0b41bc5dc4b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:25.308764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d5774-07b0-4316-a7b1-0b41bc5dc4b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:25.309036 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d5774-07b0-4316-a7b1-0b41bc5dc4b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:25.318689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d5774-07b0-4316-a7b1-0b41bc5dc4b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:25.320109 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d5774-07b0-4316-a7b1-0b41bc5dc4b3 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:25.329971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1203/2416] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:48:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:48:25.449574 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106cac3f-b2c7-4a5d-9855-1f2194ea503c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:25.450966 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-106cac3f-b2c7-4a5d-9855-1f2194ea503c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:25.462716 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106cac3f-b2c7-4a5d-9855-1f2194ea503c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:25.466281 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368896de-2f3c-470c-b09e-1b8ad4d85bb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:25.466878 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368896de-2f3c-470c-b09e-1b8ad4d85bb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:25.467151 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368896de-2f3c-470c-b09e-1b8ad4d85bb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:25.467415 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368896de-2f3c-470c-b09e-1b8ad4d85bb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:25.469014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-368896de-2f3c-470c-b09e-1b8ad4d85bb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:25.483099 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368896de-2f3c-470c-b09e-1b8ad4d85bb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:25.484707 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368896de-2f3c-470c-b09e-1b8ad4d85bb5 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:25.495545 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1214/2417] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:48:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:48:26.040576 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d5774-07b0-4316-a7b1-0b41bc5dc4b3 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:26.046601 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0b0d5774-07b0-4316-a7b1-0b41bc5dc4b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:26.059067 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d5774-07b0-4316-a7b1-0b41bc5dc4b3 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:26.064298 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2aba2-b5ac-4c43-8373-6836a58de2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:26.066315 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2aba2-b5ac-4c43-8373-6836a58de2ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:26.066845 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2aba2-b5ac-4c43-8373-6836a58de2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:26.067151 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2aba2-b5ac-4c43-8373-6836a58de2ba admin admin] PATH_INFO: `/v3/projects/8c01599e43d749f6945a0917754d5774` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:26.068385 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2aba2-b5ac-4c43-8373-6836a58de2ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c01599e43d749f6945a0917754d5774)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:26.070767 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2aba2-b5ac-4c43-8373-6836a58de2ba admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:26.211336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-2eb2aba2-b5ac-4c43-8373-6836a58de2ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c01599e43d749f6945a0917754d5774', 'request_id': 'req-2eb2aba2-b5ac-4c43-8373-6836a58de2ba'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:26.217943 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-2eb2aba2-b5ac-4c43-8373-6836a58de2ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c01599e43d749f6945a0917754d5774', 'request_id': 'req-2eb2aba2-b5ac-4c43-8373-6836a58de2ba'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:26.224104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-2eb2aba2-b5ac-4c43-8373-6836a58de2ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c01599e43d749f6945a0917754d5774', 'request_id': 'req-2eb2aba2-b5ac-4c43-8373-6836a58de2ba'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:26.288995 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-2eb2aba2-b5ac-4c43-8373-6836a58de2ba admin admin] The token cache is being invalidate because project 8c01599e43d749f6945a0917754d5774 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:48:26.289412 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-2eb2aba2-b5ac-4c43-8373-6836a58de2ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:26.290127 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-2eb2aba2-b5ac-4c43-8373-6836a58de2ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:26.291698 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1204/2418] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:48:26 2022] DELETE /identity/v3/projects/8c01599e43d749f6945a0917754d5774 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:26.295781 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368896de-2f3c-470c-b09e-1b8ad4d85bb5 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:26.306695 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368896de-2f3c-470c-b09e-1b8ad4d85bb5 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:26.369767 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920dff6d-2c99-4328-9726-f1ac658dda7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:26.375374 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920dff6d-2c99-4328-9726-f1ac658dda7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:26.375374 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920dff6d-2c99-4328-9726-f1ac658dda7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:26.375374 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920dff6d-2c99-4328-9726-f1ac658dda7e admin admin] PATH_INFO: `/v3/users/d0a32dc3d81e42288f65468cc6e964b1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:26.375374 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920dff6d-2c99-4328-9726-f1ac658dda7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0a32dc3d81e42288f65468cc6e964b1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:26.375374 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920dff6d-2c99-4328-9726-f1ac658dda7e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:26.567244 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-920dff6d-2c99-4328-9726-f1ac658dda7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0a32dc3d81e42288f65468cc6e964b1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:26.567904 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-920dff6d-2c99-4328-9726-f1ac658dda7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0a32dc3d81e42288f65468cc6e964b1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:26.568689 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-920dff6d-2c99-4328-9726-f1ac658dda7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0a32dc3d81e42288f65468cc6e964b1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:26.576638 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-920dff6d-2c99-4328-9726-f1ac658dda7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0a32dc3d81e42288f65468cc6e964b1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:26.637290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-920dff6d-2c99-4328-9726-f1ac658dda7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0a32dc3d81e42288f65468cc6e964b1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:26.665201 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1215/2419] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:48:26 2022] DELETE /identity/v3/users/d0a32dc3d81e42288f65468cc6e964b1 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:27.161238 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2aba2-b5ac-4c43-8373-6836a58de2ba admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:27.165325 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2eb2aba2-b5ac-4c43-8373-6836a58de2ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:27.224262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2aba2-b5ac-4c43-8373-6836a58de2ba admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:27.265274 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:27.295867 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571f523-d71f-4b11-92f7-9dc6f1eb5d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:27.296799 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571f523-d71f-4b11-92f7-9dc6f1eb5d1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:27.297078 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571f523-d71f-4b11-92f7-9dc6f1eb5d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:27.297384 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571f523-d71f-4b11-92f7-9dc6f1eb5d1a admin admin] PATH_INFO: `/v3/projects/2d78e651c1f5403cb7d1de8195176e6e` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:27.298542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d571f523-d71f-4b11-92f7-9dc6f1eb5d1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d78e651c1f5403cb7d1de8195176e6e)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:27.300755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d571f523-d71f-4b11-92f7-9dc6f1eb5d1a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:27.397717 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d571f523-d71f-4b11-92f7-9dc6f1eb5d1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d78e651c1f5403cb7d1de8195176e6e', 'request_id': 'req-d571f523-d71f-4b11-92f7-9dc6f1eb5d1a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:27.404436 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d571f523-d71f-4b11-92f7-9dc6f1eb5d1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d78e651c1f5403cb7d1de8195176e6e', 'request_id': 'req-d571f523-d71f-4b11-92f7-9dc6f1eb5d1a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:27.416873 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d571f523-d71f-4b11-92f7-9dc6f1eb5d1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d78e651c1f5403cb7d1de8195176e6e', 'request_id': 'req-d571f523-d71f-4b11-92f7-9dc6f1eb5d1a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:27.528117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d571f523-d71f-4b11-92f7-9dc6f1eb5d1a admin admin] The token cache is being invalidate because project 2d78e651c1f5403cb7d1de8195176e6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:48:27.528438 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d571f523-d71f-4b11-92f7-9dc6f1eb5d1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:27.529217 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d571f523-d71f-4b11-92f7-9dc6f1eb5d1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:27.530967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1205/2420] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:48:27 2022] DELETE /identity/v3/projects/2d78e651c1f5403cb7d1de8195176e6e => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:27.536556 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090df3b-93e1-41a4-a17f-aa6057a9bfdb None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:27.537717 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090df3b-93e1-41a4-a17f-aa6057a9bfdb None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:27.541302 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090df3b-93e1-41a4-a17f-aa6057a9bfdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:27.565534 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-3090df3b-93e1-41a4-a17f-aa6057a9bfdb None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:27.568520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3090df3b-93e1-41a4-a17f-aa6057a9bfdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:27.580277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:27.643217 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1216/2421] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:27.654815 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25db80-717f-4073-bbf5-95c5165630e1 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:27.654815 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25db80-717f-4073-bbf5-95c5165630e1 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:27.654815 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25db80-717f-4073-bbf5-95c5165630e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:27.674897 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-cc25db80-717f-4073-bbf5-95c5165630e1 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:27.678101 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-cc25db80-717f-4073-bbf5-95c5165630e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:27.691211 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1206/2422] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:27.692389 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090df3b-93e1-41a4-a17f-aa6057a9bfdb None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:27.704419 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090df3b-93e1-41a4-a17f-aa6057a9bfdb None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:27.715695 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a1e4d-9be1-4359-b6fc-f0f823a4e2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:27.716539 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a1e4d-9be1-4359-b6fc-f0f823a4e2a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:27.717273 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a1e4d-9be1-4359-b6fc-f0f823a4e2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:27.717273 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a1e4d-9be1-4359-b6fc-f0f823a4e2a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:27.718404 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49a1e4d-9be1-4359-b6fc-f0f823a4e2a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:48:27.718968 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49a1e4d-9be1-4359-b6fc-f0f823a4e2a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:27.720719 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49a1e4d-9be1-4359-b6fc-f0f823a4e2a6 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:27.738498 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1217/2423] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:48:27.742504 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc25db80-717f-4073-bbf5-95c5165630e1 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:27.755384 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc25db80-717f-4073-bbf5-95c5165630e1 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:27.762673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e39a8-cbbb-48e7-aecc-d514826b3173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:27.763549 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e39a8-cbbb-48e7-aecc-d514826b3173 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:27.764359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e39a8-cbbb-48e7-aecc-d514826b3173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:27.764359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e39a8-cbbb-48e7-aecc-d514826b3173 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:27.765030 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e39a8-cbbb-48e7-aecc-d514826b3173 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:27.767140 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e39a8-cbbb-48e7-aecc-d514826b3173 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:27.859912 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1207/2424] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:48:27 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:48:27.861246 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a1e4d-9be1-4359-b6fc-f0f823a4e2a6 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:27.871548 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a1e4d-9be1-4359-b6fc-f0f823a4e2a6 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:27.928751 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a21ce-c0fb-43a3-a97f-10425985deeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:27.929465 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a21ce-c0fb-43a3-a97f-10425985deeb admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:27.929749 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a21ce-c0fb-43a3-a97f-10425985deeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:27.930037 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a21ce-c0fb-43a3-a97f-10425985deeb admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:27.930747 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a21ce-c0fb-43a3-a97f-10425985deeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:27.932661 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a21ce-c0fb-43a3-a97f-10425985deeb admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:28.042807 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1218/2425] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:48:27 2022] POST /identity/v3/users => generated 288 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:48:28.044349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e39a8-cbbb-48e7-aecc-d514826b3173 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:28.058835 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e39a8-cbbb-48e7-aecc-d514826b3173 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:28.064049 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c62bd-3795-4a20-956e-f58314628139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:28.064705 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c62bd-3795-4a20-956e-f58314628139 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:28.065744 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c62bd-3795-4a20-956e-f58314628139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:28.066143 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c62bd-3795-4a20-956e-f58314628139 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:28.066719 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c62bd-3795-4a20-956e-f58314628139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:28.068333 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c62bd-3795-4a20-956e-f58314628139 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:28.081179 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1208/2426] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:28.084675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a21ce-c0fb-43a3-a97f-10425985deeb admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:28.101541 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a21ce-c0fb-43a3-a97f-10425985deeb admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:28.107608 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292949c-5b3e-46b8-8140-09ac49b6d7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:28.108220 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292949c-5b3e-46b8-8140-09ac49b6d7a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:28.108505 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292949c-5b3e-46b8-8140-09ac49b6d7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:28.108783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292949c-5b3e-46b8-8140-09ac49b6d7a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:28.109474 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292949c-5b3e-46b8-8140-09ac49b6d7a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:28.111091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292949c-5b3e-46b8-8140-09ac49b6d7a2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:28.129349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c62bd-3795-4a20-956e-f58314628139 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:28.129795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1219/2427] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:28.141276 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c62bd-3795-4a20-956e-f58314628139 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:28.147109 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef86041-33d5-497d-9a41-0f47742d9f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:28.147840 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef86041-33d5-497d-9a41-0f47742d9f44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:28.148127 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef86041-33d5-497d-9a41-0f47742d9f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:28.148523 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef86041-33d5-497d-9a41-0f47742d9f44 admin admin] PATH_INFO: `/v3/projects/61508804d4294a11aff8825c1d25b657/users/2102c960a4804973bfad9f17ec68d6d5/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:28.165340 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef86041-33d5-497d-9a41-0f47742d9f44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61508804d4294a11aff8825c1d25b657, user_id=2102c960a4804973bfad9f17ec68d6d5, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:28.168260 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef86041-33d5-497d-9a41-0f47742d9f44 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:28.273791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1209/2428] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:48:28 2022] PUT /identity/v3/projects/61508804d4294a11aff8825c1d25b657/users/2102c960a4804973bfad9f17ec68d6d5/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:28.286590 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244962e1-42ec-4301-bbd9-87f2d1b76c7c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:28.286958 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244962e1-42ec-4301-bbd9-87f2d1b76c7c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:28.287296 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244962e1-42ec-4301-bbd9-87f2d1b76c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:28.318708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-244962e1-42ec-4301-bbd9-87f2d1b76c7c None None] MFA Rules not processed for user `2102c960a4804973bfad9f17ec68d6d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:28.319933 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-244962e1-42ec-4301-bbd9-87f2d1b76c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:28.351864 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef86041-33d5-497d-9a41-0f47742d9f44 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:28.354040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-9ef86041-33d5-497d-9a41-0f47742d9f44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:28.410264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1220/2429] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:28.421116 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244962e1-42ec-4301-bbd9-87f2d1b76c7c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:28.432830 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244962e1-42ec-4301-bbd9-87f2d1b76c7c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:28.433698 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef86041-33d5-497d-9a41-0f47742d9f44 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:28.437718 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4085ac0b-4359-471d-9503-8c1a0c24650b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:28.438331 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085ac0b-4359-471d-9503-8c1a0c24650b admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:28.438749 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085ac0b-4359-471d-9503-8c1a0c24650b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:28.438864 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a8460-db3e-419e-8221-1edb3ec59afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:28.439024 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085ac0b-4359-471d-9503-8c1a0c24650b admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:28.439648 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4085ac0b-4359-471d-9503-8c1a0c24650b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:28.439842 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a8460-db3e-419e-8221-1edb3ec59afd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:28.440745 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a8460-db3e-419e-8221-1edb3ec59afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:28.441237 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a8460-db3e-419e-8221-1edb3ec59afd admin admin] PATH_INFO: `/v3/users/d298b4324a304b8e9d1deeac7420b83d` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:28.442509 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4085ac0b-4359-471d-9503-8c1a0c24650b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:28.443233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a8460-db3e-419e-8221-1edb3ec59afd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d298b4324a304b8e9d1deeac7420b83d)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:28.445311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a8460-db3e-419e-8221-1edb3ec59afd admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:28.447301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:28.448009 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:28.535715 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1221/2430] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:48:28 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:48:28.536688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4085ac0b-4359-471d-9503-8c1a0c24650b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:28.614984 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4085ac0b-4359-471d-9503-8c1a0c24650b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:28.621165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020558bb-9069-43a4-b17d-06c1390fec1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:28.621786 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020558bb-9069-43a4-b17d-06c1390fec1a admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:28.622068 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020558bb-9069-43a4-b17d-06c1390fec1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:28.622342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020558bb-9069-43a4-b17d-06c1390fec1a admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:28.623088 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020558bb-9069-43a4-b17d-06c1390fec1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:28.625080 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020558bb-9069-43a4-b17d-06c1390fec1a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:28.654321 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-107a8460-db3e-419e-8221-1edb3ec59afd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd298b4324a304b8e9d1deeac7420b83d'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:28.654970 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-107a8460-db3e-419e-8221-1edb3ec59afd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd298b4324a304b8e9d1deeac7420b83d'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:28.655564 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-107a8460-db3e-419e-8221-1edb3ec59afd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd298b4324a304b8e9d1deeac7420b83d'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:28.662676 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-107a8460-db3e-419e-8221-1edb3ec59afd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd298b4324a304b8e9d1deeac7420b83d'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:28.719338 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1222/2431] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:48:28 2022] POST /identity/v3/users => generated 295 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:48:28.723827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020558bb-9069-43a4-b17d-06c1390fec1a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:28.725586 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-020558bb-9069-43a4-b17d-06c1390fec1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:28.734486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-107a8460-db3e-419e-8221-1edb3ec59afd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd298b4324a304b8e9d1deeac7420b83d'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:28.739463 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020558bb-9069-43a4-b17d-06c1390fec1a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:28.745237 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9d324-0685-4d06-a820-53cc3594e5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:28.745854 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9d324-0685-4d06-a820-53cc3594e5e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:28.746127 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9d324-0685-4d06-a820-53cc3594e5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:28.746407 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9d324-0685-4d06-a820-53cc3594e5e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:28.747054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f9d324-0685-4d06-a820-53cc3594e5e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:28.748726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f9d324-0685-4d06-a820-53cc3594e5e9 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:28.760622 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1210/2432] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:48:28 2022] DELETE /identity/v3/users/d298b4324a304b8e9d1deeac7420b83d => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:48:28.770244 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1223/2433] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:48:28.770403 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391c233-214b-47e5-a3bf-ede21b3ad7f8 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:28.770403 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391c233-214b-47e5-a3bf-ede21b3ad7f8 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:28.770594 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9d324-0685-4d06-a820-53cc3594e5e9 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:28.770695 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391c233-214b-47e5-a3bf-ede21b3ad7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:28.779340 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9d324-0685-4d06-a820-53cc3594e5e9 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:28.789881 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-f391c233-214b-47e5-a3bf-ede21b3ad7f8 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:28.791202 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f391c233-214b-47e5-a3bf-ede21b3ad7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:28.849267 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c2495-4a8a-4eaa-9f6d-a43c340c3b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:28.849942 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c2495-4a8a-4eaa-9f6d-a43c340c3b79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:28.850267 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c2495-4a8a-4eaa-9f6d-a43c340c3b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:28.850791 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c2495-4a8a-4eaa-9f6d-a43c340c3b79 admin admin] PATH_INFO: `/v3/projects/decf39b61a19480dba5e4fd3c4f62e1d/users/9acb36ac474449aa896879260d5bfccc/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:28.861534 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1211/2434] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:48:28.869059 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c2495-4a8a-4eaa-9f6d-a43c340c3b79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=decf39b61a19480dba5e4fd3c4f62e1d, user_id=9acb36ac474449aa896879260d5bfccc, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:28.872406 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c2495-4a8a-4eaa-9f6d-a43c340c3b79 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:28.873990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391c233-214b-47e5-a3bf-ede21b3ad7f8 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:28.875015 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f391c233-214b-47e5-a3bf-ede21b3ad7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:28.934836 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391c233-214b-47e5-a3bf-ede21b3ad7f8 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:28.938701 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63bca4d-1e94-4aba-b49b-0be5d167339e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:28.939439 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63bca4d-1e94-4aba-b49b-0be5d167339e service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:28.947223 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63bca4d-1e94-4aba-b49b-0be5d167339e service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:28.947223 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63bca4d-1e94-4aba-b49b-0be5d167339e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:28.953904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63bca4d-1e94-4aba-b49b-0be5d167339e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:28.955635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63bca4d-1e94-4aba-b49b-0be5d167339e service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:28.969377 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1224/2435] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:48:28 2022] PUT /identity/v3/projects/decf39b61a19480dba5e4fd3c4f62e1d/users/9acb36ac474449aa896879260d5bfccc/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:48:28.973478 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1212/2436] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:48:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:48:28.973656 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c2495-4a8a-4eaa-9f6d-a43c340c3b79 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:29.035279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:29.052928 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c2495-4a8a-4eaa-9f6d-a43c340c3b79 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:29.082048 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41642126-856f-4ca4-a05e-479ab57ad3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:29.084153 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41642126-856f-4ca4-a05e-479ab57ad3e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:29.084153 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41642126-856f-4ca4-a05e-479ab57ad3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:29.084153 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41642126-856f-4ca4-a05e-479ab57ad3e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:29.084153 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41642126-856f-4ca4-a05e-479ab57ad3e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:29.085913 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41642126-856f-4ca4-a05e-479ab57ad3e5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:29.100935 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1225/2437] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:29.102396 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63bca4d-1e94-4aba-b49b-0be5d167339e service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:29.121007 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63bca4d-1e94-4aba-b49b-0be5d167339e service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:29.129312 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4597abd-452b-4cc3-9eeb-78765a8249eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:29.129995 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4597abd-452b-4cc3-9eeb-78765a8249eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:29.130275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4597abd-452b-4cc3-9eeb-78765a8249eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:29.130550 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4597abd-452b-4cc3-9eeb-78765a8249eb admin admin] PATH_INFO: `/v3/projects/decf39b61a19480dba5e4fd3c4f62e1d/users/9acb36ac474449aa896879260d5bfccc/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:29.132666 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4597abd-452b-4cc3-9eeb-78765a8249eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=decf39b61a19480dba5e4fd3c4f62e1d, user_id=9acb36ac474449aa896879260d5bfccc, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:29.137259 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4597abd-452b-4cc3-9eeb-78765a8249eb admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:29.155190 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1213/2438] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:48:29 2022] PUT /identity/v3/projects/decf39b61a19480dba5e4fd3c4f62e1d/users/9acb36ac474449aa896879260d5bfccc/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:29.166221 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de8d24-f088-4dc6-b018-ca1727f95d31 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:29.166537 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de8d24-f088-4dc6-b018-ca1727f95d31 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:29.166837 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de8d24-f088-4dc6-b018-ca1727f95d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:29.215455 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-30de8d24-f088-4dc6-b018-ca1727f95d31 None None] MFA Rules not processed for user `9acb36ac474449aa896879260d5bfccc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:29.216692 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-30de8d24-f088-4dc6-b018-ca1727f95d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:29.309769 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1226/2439] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:29.321869 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4597abd-452b-4cc3-9eeb-78765a8249eb admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:29.324606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d4597abd-452b-4cc3-9eeb-78765a8249eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:29.339899 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:29.432378 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4597abd-452b-4cc3-9eeb-78765a8249eb admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:29.437576 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399cb92d-983b-4687-b5f4-1f2b7b34d6cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:29.438199 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cb92d-983b-4687-b5f4-1f2b7b34d6cf service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:29.438477 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cb92d-983b-4687-b5f4-1f2b7b34d6cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:29.438745 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cb92d-983b-4687-b5f4-1f2b7b34d6cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:29.441392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-399cb92d-983b-4687-b5f4-1f2b7b34d6cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:29.513797 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399cb92d-983b-4687-b5f4-1f2b7b34d6cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:29.515237 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399cb92d-983b-4687-b5f4-1f2b7b34d6cf service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:29.525307 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1214/2440] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:48:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:48:29.614870 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de8d24-f088-4dc6-b018-ca1727f95d31 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:29.616321 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-30de8d24-f088-4dc6-b018-ca1727f95d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:29.743858 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de8d24-f088-4dc6-b018-ca1727f95d31 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:29.743858 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46efb373-bb24-46f4-a346-a2f5d3e9eace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:29.743858 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46efb373-bb24-46f4-a346-a2f5d3e9eace admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:29.743858 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46efb373-bb24-46f4-a346-a2f5d3e9eace admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:29.744542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46efb373-bb24-46f4-a346-a2f5d3e9eace admin admin] PATH_INFO: `/v3/projects/ea9f95038e5d4ac3aa33ebdce5f63055` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:29.745877 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46efb373-bb24-46f4-a346-a2f5d3e9eace admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea9f95038e5d4ac3aa33ebdce5f63055)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:29.750449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46efb373-bb24-46f4-a346-a2f5d3e9eace admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:29.992697 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-46efb373-bb24-46f4-a346-a2f5d3e9eace admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea9f95038e5d4ac3aa33ebdce5f63055', 'request_id': 'req-46efb373-bb24-46f4-a346-a2f5d3e9eace'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:30.027350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399cb92d-983b-4687-b5f4-1f2b7b34d6cf service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:30.028785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-399cb92d-983b-4687-b5f4-1f2b7b34d6cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:30.043404 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399cb92d-983b-4687-b5f4-1f2b7b34d6cf service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:30.088849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-46efb373-bb24-46f4-a346-a2f5d3e9eace admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea9f95038e5d4ac3aa33ebdce5f63055', 'request_id': 'req-46efb373-bb24-46f4-a346-a2f5d3e9eace'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:30.095708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-46efb373-bb24-46f4-a346-a2f5d3e9eace admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea9f95038e5d4ac3aa33ebdce5f63055', 'request_id': 'req-46efb373-bb24-46f4-a346-a2f5d3e9eace'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:30.104180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f018322-2a7b-4d8a-a010-35a17fd0a64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:30.104739 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:30.104881 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f018322-2a7b-4d8a-a010-35a17fd0a64e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:30.105167 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f018322-2a7b-4d8a-a010-35a17fd0a64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:30.105440 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f018322-2a7b-4d8a-a010-35a17fd0a64e admin admin] PATH_INFO: `/v3/users/b3e5a78c3b6c48d9a4aed7d36fd54736` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:30.106675 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f018322-2a7b-4d8a-a010-35a17fd0a64e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3e5a78c3b6c48d9a4aed7d36fd54736)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:30.108743 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f018322-2a7b-4d8a-a010-35a17fd0a64e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:30.108968 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-46efb373-bb24-46f4-a346-a2f5d3e9eace admin admin] The token cache is being invalidate because project ea9f95038e5d4ac3aa33ebdce5f63055 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:48:30.109287 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-46efb373-bb24-46f4-a346-a2f5d3e9eace admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:30.109920 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-46efb373-bb24-46f4-a346-a2f5d3e9eace admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:30.111552 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1227/2441] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:48:29 2022] DELETE /identity/v3/projects/ea9f95038e5d4ac3aa33ebdce5f63055 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:30.119292 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46efb373-bb24-46f4-a346-a2f5d3e9eace admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:30.119292 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-46efb373-bb24-46f4-a346-a2f5d3e9eace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:30.131010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46efb373-bb24-46f4-a346-a2f5d3e9eace admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:30.136421 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1787b5-88da-4799-857d-6b69b327acbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:30.137086 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1787b5-88da-4799-857d-6b69b327acbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:30.137394 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1787b5-88da-4799-857d-6b69b327acbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:30.137663 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1787b5-88da-4799-857d-6b69b327acbd admin admin] PATH_INFO: `/v3/users/ffbe3433c0c14606b5161246ea956473` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:30.138823 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1787b5-88da-4799-857d-6b69b327acbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffbe3433c0c14606b5161246ea956473)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:30.140747 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1787b5-88da-4799-857d-6b69b327acbd admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:30.344220 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5f018322-2a7b-4d8a-a010-35a17fd0a64e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3e5a78c3b6c48d9a4aed7d36fd54736'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:30.345190 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5f018322-2a7b-4d8a-a010-35a17fd0a64e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3e5a78c3b6c48d9a4aed7d36fd54736'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:30.345865 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5f018322-2a7b-4d8a-a010-35a17fd0a64e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3e5a78c3b6c48d9a4aed7d36fd54736'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:30.347853 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-4b1787b5-88da-4799-857d-6b69b327acbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffbe3433c0c14606b5161246ea956473'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:30.348502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-4b1787b5-88da-4799-857d-6b69b327acbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffbe3433c0c14606b5161246ea956473'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:30.349105 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-4b1787b5-88da-4799-857d-6b69b327acbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffbe3433c0c14606b5161246ea956473'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:30.355338 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5f018322-2a7b-4d8a-a010-35a17fd0a64e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3e5a78c3b6c48d9a4aed7d36fd54736'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:30.357456 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-4b1787b5-88da-4799-857d-6b69b327acbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffbe3433c0c14606b5161246ea956473'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:30.431467 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5f018322-2a7b-4d8a-a010-35a17fd0a64e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3e5a78c3b6c48d9a4aed7d36fd54736'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:30.433814 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:30.450385 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1215/2442] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:48:30 2022] DELETE /identity/v3/users/b3e5a78c3b6c48d9a4aed7d36fd54736 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:48:30.494660 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-4b1787b5-88da-4799-857d-6b69b327acbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffbe3433c0c14606b5161246ea956473'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:30.517391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1228/2443] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:48:30 2022] DELETE /identity/v3/users/ffbe3433c0c14606b5161246ea956473 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:48:30.699678 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f018322-2a7b-4d8a-a010-35a17fd0a64e admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:30.702628 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5f018322-2a7b-4d8a-a010-35a17fd0a64e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:30.811084 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f018322-2a7b-4d8a-a010-35a17fd0a64e admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:30.867736 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d3654c-469a-48e7-9067-32bdf9e57461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:30.868434 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3654c-469a-48e7-9067-32bdf9e57461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:30.868653 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3654c-469a-48e7-9067-32bdf9e57461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:30.868925 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3654c-469a-48e7-9067-32bdf9e57461 admin admin] PATH_INFO: `/v3/projects/2841a629bc1f4885a356218c2d0721b7` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:30.870068 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d3654c-469a-48e7-9067-32bdf9e57461 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2841a629bc1f4885a356218c2d0721b7)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:30.877483 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d3654c-469a-48e7-9067-32bdf9e57461 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:30.919649 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1787b5-88da-4799-857d-6b69b327acbd admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:30.920404 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4b1787b5-88da-4799-857d-6b69b327acbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:30.936301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1787b5-88da-4799-857d-6b69b327acbd admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:30.941722 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b6cdcd-d995-416c-a3e7-743d3d3f28b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:30.946973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6cdcd-d995-416c-a3e7-743d3d3f28b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:30.946973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6cdcd-d995-416c-a3e7-743d3d3f28b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:30.946973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6cdcd-d995-416c-a3e7-743d3d3f28b8 admin admin] PATH_INFO: `/v3/projects/1a7f46cdc0314386950ebb7afba1455f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:30.946973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b6cdcd-d995-416c-a3e7-743d3d3f28b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a7f46cdc0314386950ebb7afba1455f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:30.946973 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b6cdcd-d995-416c-a3e7-743d3d3f28b8 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:30.975413 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f8d3654c-469a-48e7-9067-32bdf9e57461 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2841a629bc1f4885a356218c2d0721b7', 'request_id': 'req-f8d3654c-469a-48e7-9067-32bdf9e57461'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:30.982133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f8d3654c-469a-48e7-9067-32bdf9e57461 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2841a629bc1f4885a356218c2d0721b7', 'request_id': 'req-f8d3654c-469a-48e7-9067-32bdf9e57461'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:30.988935 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f8d3654c-469a-48e7-9067-32bdf9e57461 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2841a629bc1f4885a356218c2d0721b7', 'request_id': 'req-f8d3654c-469a-48e7-9067-32bdf9e57461'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:31.152718 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f8d3654c-469a-48e7-9067-32bdf9e57461 admin admin] The token cache is being invalidate because project 2841a629bc1f4885a356218c2d0721b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:48:31.153012 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f8d3654c-469a-48e7-9067-32bdf9e57461 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:31.153682 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f8d3654c-469a-48e7-9067-32bdf9e57461 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:31.155333 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1216/2444] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:48:30 2022] DELETE /identity/v3/projects/2841a629bc1f4885a356218c2d0721b7 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:31.161119 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45337b-0e0c-47bf-a3cf-c95b4632f230 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:31.161430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45337b-0e0c-47bf-a3cf-c95b4632f230 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:31.161705 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45337b-0e0c-47bf-a3cf-c95b4632f230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:31.171200 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a1b6cdcd-d995-416c-a3e7-743d3d3f28b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a7f46cdc0314386950ebb7afba1455f', 'request_id': 'req-a1b6cdcd-d995-416c-a3e7-743d3d3f28b8'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:31.187908 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-db45337b-0e0c-47bf-a3cf-c95b4632f230 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:31.191458 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-db45337b-0e0c-47bf-a3cf-c95b4632f230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:31.250684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1217/2445] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:31.259706 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ad9a1-b384-4fa7-8e19-6857687982f8 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:31.259980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ad9a1-b384-4fa7-8e19-6857687982f8 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:31.260254 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ad9a1-b384-4fa7-8e19-6857687982f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:31.266146 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a1b6cdcd-d995-416c-a3e7-743d3d3f28b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a7f46cdc0314386950ebb7afba1455f', 'request_id': 'req-a1b6cdcd-d995-416c-a3e7-743d3d3f28b8'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:31.273968 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a1b6cdcd-d995-416c-a3e7-743d3d3f28b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a7f46cdc0314386950ebb7afba1455f', 'request_id': 'req-a1b6cdcd-d995-416c-a3e7-743d3d3f28b8'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:31.283061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-471ad9a1-b384-4fa7-8e19-6857687982f8 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:31.284314 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-471ad9a1-b384-4fa7-8e19-6857687982f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:31.287521 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a1b6cdcd-d995-416c-a3e7-743d3d3f28b8 admin admin] The token cache is being invalidate because project 1a7f46cdc0314386950ebb7afba1455f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:48:31.287902 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a1b6cdcd-d995-416c-a3e7-743d3d3f28b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:31.288638 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a1b6cdcd-d995-416c-a3e7-743d3d3f28b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:31.290479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1229/2446] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:48:30 2022] DELETE /identity/v3/projects/1a7f46cdc0314386950ebb7afba1455f => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:31.296364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b6cdcd-d995-416c-a3e7-743d3d3f28b8 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:31.296498 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1218/2447] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:48:31.296605 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a1b6cdcd-d995-416c-a3e7-743d3d3f28b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:31.299348 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ad9a1-b384-4fa7-8e19-6857687982f8 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:31.308676 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ad9a1-b384-4fa7-8e19-6857687982f8 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:31.309513 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b6cdcd-d995-416c-a3e7-743d3d3f28b8 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:31.319271 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d489d-1627-4e48-acbe-2f16c3027a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:31.319271 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d489d-1627-4e48-acbe-2f16c3027a8a admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:31.319271 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d489d-1627-4e48-acbe-2f16c3027a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:31.319271 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d489d-1627-4e48-acbe-2f16c3027a8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:31.319731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434444d5-52eb-4f91-8b82-b65d97ba5856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:31.319731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434444d5-52eb-4f91-8b82-b65d97ba5856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:31.319906 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d489d-1627-4e48-acbe-2f16c3027a8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:48:31.319991 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434444d5-52eb-4f91-8b82-b65d97ba5856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:31.320067 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d489d-1627-4e48-acbe-2f16c3027a8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:31.320163 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434444d5-52eb-4f91-8b82-b65d97ba5856 admin admin] PATH_INFO: `/v3/users/0f146715642a482f837d4b46d320b751` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:31.320163 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434444d5-52eb-4f91-8b82-b65d97ba5856 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f146715642a482f837d4b46d320b751)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:31.320318 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d489d-1627-4e48-acbe-2f16c3027a8a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:31.320400 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434444d5-52eb-4f91-8b82-b65d97ba5856 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:31.330097 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1219/2448] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:48:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:48:31.334875 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d489d-1627-4e48-acbe-2f16c3027a8a admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:31.336626 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-3a5d489d-1627-4e48-acbe-2f16c3027a8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:31.349131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d489d-1627-4e48-acbe-2f16c3027a8a admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:31.354148 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a5462-5b20-4f05-abfa-b99e678f1585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:31.354764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a5462-5b20-4f05-abfa-b99e678f1585 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:31.355042 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a5462-5b20-4f05-abfa-b99e678f1585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:31.355312 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a5462-5b20-4f05-abfa-b99e678f1585 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:31.355922 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a5462-5b20-4f05-abfa-b99e678f1585 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:31.357841 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a5462-5b20-4f05-abfa-b99e678f1585 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:31.364450 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:31.464537 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1220/2449] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:48:31 2022] POST /identity/v3/projects => generated 371 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:48:31.465599 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a5462-5b20-4f05-abfa-b99e678f1585 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:31.515368 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-434444d5-52eb-4f91-8b82-b65d97ba5856 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f146715642a482f837d4b46d320b751'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:31.516035 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-434444d5-52eb-4f91-8b82-b65d97ba5856 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f146715642a482f837d4b46d320b751'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:31.516647 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-434444d5-52eb-4f91-8b82-b65d97ba5856 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f146715642a482f837d4b46d320b751'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:31.519436 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=74,keystone:DELETE=24,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:31.524858 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-434444d5-52eb-4f91-8b82-b65d97ba5856 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f146715642a482f837d4b46d320b751'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:31.528288 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a5462-5b20-4f05-abfa-b99e678f1585 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:31.530530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32031b12-40e0-4d9b-8586-a472ee554b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:31.531113 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32031b12-40e0-4d9b-8586-a472ee554b64 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:31.531386 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32031b12-40e0-4d9b-8586-a472ee554b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:31.531655 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32031b12-40e0-4d9b-8586-a472ee554b64 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:31.532270 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32031b12-40e0-4d9b-8586-a472ee554b64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:31.534331 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32031b12-40e0-4d9b-8586-a472ee554b64 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:31.631436 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-434444d5-52eb-4f91-8b82-b65d97ba5856 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f146715642a482f837d4b46d320b751'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:31.651436 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1230/2450] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:48:31 2022] DELETE /identity/v3/users/0f146715642a482f837d4b46d320b751 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:31.651753 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1221/2451] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:48:31 2022] POST /identity/v3/users => generated 286 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:48:31.655623 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434444d5-52eb-4f91-8b82-b65d97ba5856 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:31.657026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-434444d5-52eb-4f91-8b82-b65d97ba5856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:31.724183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434444d5-52eb-4f91-8b82-b65d97ba5856 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:31.729557 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0396f0c-54cd-45da-81af-1049d421f0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:31.730168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0396f0c-54cd-45da-81af-1049d421f0c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:31.730445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0396f0c-54cd-45da-81af-1049d421f0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:31.730715 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0396f0c-54cd-45da-81af-1049d421f0c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:31.731666 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0396f0c-54cd-45da-81af-1049d421f0c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:31.733974 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0396f0c-54cd-45da-81af-1049d421f0c7 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:31.746631 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1231/2452] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:48:31.750885 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32031b12-40e0-4d9b-8586-a472ee554b64 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:31.761728 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32031b12-40e0-4d9b-8586-a472ee554b64 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:31.766291 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd0bfa0-4131-4ef6-8196-90789b33b1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:31.766888 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0bfa0-4131-4ef6-8196-90789b33b1f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:31.767167 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0bfa0-4131-4ef6-8196-90789b33b1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:31.767453 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0bfa0-4131-4ef6-8196-90789b33b1f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:31.768073 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd0bfa0-4131-4ef6-8196-90789b33b1f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:31.769732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd0bfa0-4131-4ef6-8196-90789b33b1f4 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:31.779605 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1222/2453] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:31.784444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0396f0c-54cd-45da-81af-1049d421f0c7 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:31.793951 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0396f0c-54cd-45da-81af-1049d421f0c7 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:31.798641 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d0fee-4114-4f36-99f1-aba93cb146d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:31.799276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d0fee-4114-4f36-99f1-aba93cb146d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:31.799563 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d0fee-4114-4f36-99f1-aba93cb146d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:31.799839 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d0fee-4114-4f36-99f1-aba93cb146d0 admin admin] PATH_INFO: `/v3/projects/27e36fc341ec4f369fa269b4585d8c9d/users/7a669e81f3994fdea127294c43913bdc/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:31.815271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2d0fee-4114-4f36-99f1-aba93cb146d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27e36fc341ec4f369fa269b4585d8c9d, user_id=7a669e81f3994fdea127294c43913bdc, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:31.818891 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2d0fee-4114-4f36-99f1-aba93cb146d0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:31.896514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1232/2454] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:48:31 2022] PUT /identity/v3/projects/27e36fc341ec4f369fa269b4585d8c9d/users/7a669e81f3994fdea127294c43913bdc/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:31.907313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea321025-f791-443f-b355-68cf50e68304 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:31.907676 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea321025-f791-443f-b355-68cf50e68304 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:31.908027 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea321025-f791-443f-b355-68cf50e68304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:31.939490 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-ea321025-f791-443f-b355-68cf50e68304 None None] MFA Rules not processed for user `7a669e81f3994fdea127294c43913bdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:31.940104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ea321025-f791-443f-b355-68cf50e68304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:32.012028 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1223/2455] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:48:32.023929 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d0fee-4114-4f36-99f1-aba93cb146d0 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:32.029182 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1b2d0fee-4114-4f36-99f1-aba93cb146d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:32.097608 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d0fee-4114-4f36-99f1-aba93cb146d0 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:32.104718 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dbcfc9-c753-43e4-9b0d-604954a7bd99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:32.105580 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbcfc9-c753-43e4-9b0d-604954a7bd99 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:32.105970 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbcfc9-c753-43e4-9b0d-604954a7bd99 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:32.106425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbcfc9-c753-43e4-9b0d-604954a7bd99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:32.108647 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea321025-f791-443f-b355-68cf50e68304 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:32.111244 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ea321025-f791-443f-b355-68cf50e68304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:32.120157 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dbcfc9-c753-43e4-9b0d-604954a7bd99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:32.123138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dbcfc9-c753-43e4-9b0d-604954a7bd99 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:32.140876 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1233/2456] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:48:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:48:32.198452 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea321025-f791-443f-b355-68cf50e68304 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:32.203065 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb45b8f-7a76-4d9d-9a03-346141a99b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:32.203697 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb45b8f-7a76-4d9d-9a03-346141a99b9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:32.203974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb45b8f-7a76-4d9d-9a03-346141a99b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:32.204280 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb45b8f-7a76-4d9d-9a03-346141a99b9c admin admin] PATH_INFO: `/v3/projects/e83a1cb21e1d4d2880d7ffce97247a96` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:32.205419 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb45b8f-7a76-4d9d-9a03-346141a99b9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e83a1cb21e1d4d2880d7ffce97247a96)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:32.207961 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb45b8f-7a76-4d9d-9a03-346141a99b9c admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:32.276875 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:32.317364 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-feb45b8f-7a76-4d9d-9a03-346141a99b9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e83a1cb21e1d4d2880d7ffce97247a96', 'request_id': 'req-feb45b8f-7a76-4d9d-9a03-346141a99b9c'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:32.323322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-feb45b8f-7a76-4d9d-9a03-346141a99b9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e83a1cb21e1d4d2880d7ffce97247a96', 'request_id': 'req-feb45b8f-7a76-4d9d-9a03-346141a99b9c'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:32.328666 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-feb45b8f-7a76-4d9d-9a03-346141a99b9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e83a1cb21e1d4d2880d7ffce97247a96', 'request_id': 'req-feb45b8f-7a76-4d9d-9a03-346141a99b9c'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:32.402774 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-feb45b8f-7a76-4d9d-9a03-346141a99b9c admin admin] The token cache is being invalidate because project e83a1cb21e1d4d2880d7ffce97247a96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:48:32.403430 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-feb45b8f-7a76-4d9d-9a03-346141a99b9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:32.403991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-feb45b8f-7a76-4d9d-9a03-346141a99b9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:32.405919 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1224/2457] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:48:32 2022] DELETE /identity/v3/projects/e83a1cb21e1d4d2880d7ffce97247a96 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:32.412859 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffce49-af2c-46e0-ab3f-f944fd116604 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:32.418107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffce49-af2c-46e0-ab3f-f944fd116604 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:32.418107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffce49-af2c-46e0-ab3f-f944fd116604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:32.435162 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-79ffce49-af2c-46e0-ab3f-f944fd116604 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:32.436537 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-79ffce49-af2c-46e0-ab3f-f944fd116604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:32.457720 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:32.506277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1234/2458] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:32.514275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a913b5c-5334-49d2-a9d0-a4ed5ef314f4 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:32.514839 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a913b5c-5334-49d2-a9d0-a4ed5ef314f4 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:32.515004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a913b5c-5334-49d2-a9d0-a4ed5ef314f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:32.535589 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-7a913b5c-5334-49d2-a9d0-a4ed5ef314f4 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:32.536790 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-7a913b5c-5334-49d2-a9d0-a4ed5ef314f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:32.544630 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1225/2459] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:32.549863 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffce49-af2c-46e0-ab3f-f944fd116604 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:32.558475 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffce49-af2c-46e0-ab3f-f944fd116604 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:32.563028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5310bd-6701-4b6e-a2ff-16f269de6226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:32.563625 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5310bd-6701-4b6e-a2ff-16f269de6226 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:32.563899 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5310bd-6701-4b6e-a2ff-16f269de6226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:32.564167 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5310bd-6701-4b6e-a2ff-16f269de6226 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:32.564808 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5310bd-6701-4b6e-a2ff-16f269de6226 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:48:32.565111 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5310bd-6701-4b6e-a2ff-16f269de6226 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:32.566771 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5310bd-6701-4b6e-a2ff-16f269de6226 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:32.577994 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1235/2460] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:48:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:48:32.585213 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a913b5c-5334-49d2-a9d0-a4ed5ef314f4 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:32.594906 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a913b5c-5334-49d2-a9d0-a4ed5ef314f4 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:32.599787 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b2052a-0352-4c77-91f4-9d3b1928e4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:32.600383 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2052a-0352-4c77-91f4-9d3b1928e4c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:32.600658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2052a-0352-4c77-91f4-9d3b1928e4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:32.600925 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2052a-0352-4c77-91f4-9d3b1928e4c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:32.601620 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b2052a-0352-4c77-91f4-9d3b1928e4c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:32.603802 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b2052a-0352-4c77-91f4-9d3b1928e4c9 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:32.702451 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1226/2461] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:48:32 2022] POST /identity/v3/projects => generated 391 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:48:32.706737 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5310bd-6701-4b6e-a2ff-16f269de6226 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:32.717659 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5310bd-6701-4b6e-a2ff-16f269de6226 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:32.778290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638dc90f-8d81-4baa-b644-390d8c91f755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:32.778898 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dc90f-8d81-4baa-b644-390d8c91f755 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:32.779176 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dc90f-8d81-4baa-b644-390d8c91f755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:32.779449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dc90f-8d81-4baa-b644-390d8c91f755 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:32.780087 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638dc90f-8d81-4baa-b644-390d8c91f755 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:32.782054 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638dc90f-8d81-4baa-b644-390d8c91f755 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:32.854415 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1236/2462] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:48:32 2022] POST /identity/v3/users => generated 296 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:48:32.855705 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b2052a-0352-4c77-91f4-9d3b1928e4c9 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:32.868700 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b2052a-0352-4c77-91f4-9d3b1928e4c9 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:32.875332 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6dd382-1c89-4d69-8a2f-32f97396dc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:32.875936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dd382-1c89-4d69-8a2f-32f97396dc7a admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:32.876215 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dd382-1c89-4d69-8a2f-32f97396dc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:32.876508 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dd382-1c89-4d69-8a2f-32f97396dc7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:32.877235 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6dd382-1c89-4d69-8a2f-32f97396dc7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:32.886026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6dd382-1c89-4d69-8a2f-32f97396dc7a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:32.889440 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1227/2463] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:32.893445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638dc90f-8d81-4baa-b644-390d8c91f755 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:32.903738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638dc90f-8d81-4baa-b644-390d8c91f755 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:32.908503 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea57793-ba4d-4361-8dc6-77d60d926f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:32.909094 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea57793-ba4d-4361-8dc6-77d60d926f02 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:32.909405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea57793-ba4d-4361-8dc6-77d60d926f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:32.914712 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea57793-ba4d-4361-8dc6-77d60d926f02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:32.914712 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea57793-ba4d-4361-8dc6-77d60d926f02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:32.914712 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea57793-ba4d-4361-8dc6-77d60d926f02 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:32.922213 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1237/2464] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:32.926893 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6dd382-1c89-4d69-8a2f-32f97396dc7a admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:32.936807 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6dd382-1c89-4d69-8a2f-32f97396dc7a admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:32.941499 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d11ef-a3f9-4665-8a5e-b5f0bfc39937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:32.946322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d11ef-a3f9-4665-8a5e-b5f0bfc39937 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:32.946322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d11ef-a3f9-4665-8a5e-b5f0bfc39937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:32.946322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d11ef-a3f9-4665-8a5e-b5f0bfc39937 admin admin] PATH_INFO: `/v3/projects/833ae0a6ab4e4990ac713489b3a2a8f4/users/377c22bb511141a7a09d8e1a2b9b0f2b/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:32.957880 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d11ef-a3f9-4665-8a5e-b5f0bfc39937 admin admin] RBAC: Authorizing `identity:create_grant(project_id=833ae0a6ab4e4990ac713489b3a2a8f4, user_id=377c22bb511141a7a09d8e1a2b9b0f2b, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:32.960491 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d11ef-a3f9-4665-8a5e-b5f0bfc39937 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:33.072231 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1228/2465] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:48:32 2022] PUT /identity/v3/projects/833ae0a6ab4e4990ac713489b3a2a8f4/users/377c22bb511141a7a09d8e1a2b9b0f2b/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:33.095477 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb299d0-7727-46f0-81c0-020d2453f68c None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:33.095986 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb299d0-7727-46f0-81c0-020d2453f68c None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:33.096086 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb299d0-7727-46f0-81c0-020d2453f68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:33.124986 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-efb299d0-7727-46f0-81c0-020d2453f68c None None] MFA Rules not processed for user `377c22bb511141a7a09d8e1a2b9b0f2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:33.126221 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-efb299d0-7727-46f0-81c0-020d2453f68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:33.188879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1238/2466] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:33 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:33.193890 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d11ef-a3f9-4665-8a5e-b5f0bfc39937 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:33.207363 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:33.255360 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d11ef-a3f9-4665-8a5e-b5f0bfc39937 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:33.259987 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2e46a-e6f8-4c52-a3f2-288214e827f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:33.260585 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2e46a-e6f8-4c52-a3f2-288214e827f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:33.260871 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2e46a-e6f8-4c52-a3f2-288214e827f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:33.261157 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2e46a-e6f8-4c52-a3f2-288214e827f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:33.261784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc2e46a-e6f8-4c52-a3f2-288214e827f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:33.263779 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc2e46a-e6f8-4c52-a3f2-288214e827f6 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:33.363905 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1229/2467] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:48:33 2022] POST /identity/v3/projects => generated 393 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:48:33.368092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb299d0-7727-46f0-81c0-020d2453f68c None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:33.378667 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb299d0-7727-46f0-81c0-020d2453f68c None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:33.395203 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:33.434928 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec5578-3ec3-4d2b-a354-265fe0073e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:33.435539 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec5578-3ec3-4d2b-a354-265fe0073e9d admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:33.435819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec5578-3ec3-4d2b-a354-265fe0073e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:33.436088 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec5578-3ec3-4d2b-a354-265fe0073e9d admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:33.436716 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec5578-3ec3-4d2b-a354-265fe0073e9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:33.438675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec5578-3ec3-4d2b-a354-265fe0073e9d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:33.551596 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1239/2468] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:48:33 2022] POST /identity/v3/users => generated 303 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:48:33.551708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2e46a-e6f8-4c52-a3f2-288214e827f6 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:33.560877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2e46a-e6f8-4c52-a3f2-288214e827f6 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:33.565596 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b60302-8341-4115-8585-5377221bd5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:33.566694 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b60302-8341-4115-8585-5377221bd5e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:33.566969 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b60302-8341-4115-8585-5377221bd5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:33.567265 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b60302-8341-4115-8585-5377221bd5e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:33.567910 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b60302-8341-4115-8585-5377221bd5e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:33.569525 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b60302-8341-4115-8585-5377221bd5e3 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:33.580353 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1230/2469] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:33.585270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec5578-3ec3-4d2b-a354-265fe0073e9d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:33.595634 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec5578-3ec3-4d2b-a354-265fe0073e9d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:33.600476 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eff2dd-d391-42a6-a0e1-a45b7b615fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:33.601204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eff2dd-d391-42a6-a0e1-a45b7b615fc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:33.601492 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eff2dd-d391-42a6-a0e1-a45b7b615fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:33.601771 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eff2dd-d391-42a6-a0e1-a45b7b615fc1 admin admin] PATH_INFO: `/v3/projects/a2f4c25ce1c34cb4bbde376be503d54d/users/f69b7214196546e1ab79d2012a426a66/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:33.617045 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eff2dd-d391-42a6-a0e1-a45b7b615fc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2f4c25ce1c34cb4bbde376be503d54d, user_id=f69b7214196546e1ab79d2012a426a66, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:33.619723 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eff2dd-d391-42a6-a0e1-a45b7b615fc1 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:33.689040 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1240/2470] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:48:33 2022] PUT /identity/v3/projects/a2f4c25ce1c34cb4bbde376be503d54d/users/f69b7214196546e1ab79d2012a426a66/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:33.693813 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b60302-8341-4115-8585-5377221bd5e3 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:33.703693 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b60302-8341-4115-8585-5377221bd5e3 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:33.758548 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7469f4c-3087-4f23-a1b8-ffe92f86e409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:33.759163 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7469f4c-3087-4f23-a1b8-ffe92f86e409 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:33.759442 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7469f4c-3087-4f23-a1b8-ffe92f86e409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:33.759713 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7469f4c-3087-4f23-a1b8-ffe92f86e409 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:33.760351 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7469f4c-3087-4f23-a1b8-ffe92f86e409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:33.761980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7469f4c-3087-4f23-a1b8-ffe92f86e409 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:33.772348 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1231/2471] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:33 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:33.776532 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eff2dd-d391-42a6-a0e1-a45b7b615fc1 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:33.787033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eff2dd-d391-42a6-a0e1-a45b7b615fc1 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:33.791510 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f775f-c027-4eab-886a-f36a1ebb28d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:33.792146 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f775f-c027-4eab-886a-f36a1ebb28d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:33.792420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f775f-c027-4eab-886a-f36a1ebb28d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:33.792687 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62f775f-c027-4eab-886a-f36a1ebb28d7 admin admin] PATH_INFO: `/v3/projects/a2f4c25ce1c34cb4bbde376be503d54d/users/f69b7214196546e1ab79d2012a426a66/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:33.794560 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62f775f-c027-4eab-886a-f36a1ebb28d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2f4c25ce1c34cb4bbde376be503d54d, user_id=f69b7214196546e1ab79d2012a426a66, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:33.797169 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62f775f-c027-4eab-886a-f36a1ebb28d7 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:33.807542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1241/2472] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:48:33 2022] PUT /identity/v3/projects/a2f4c25ce1c34cb4bbde376be503d54d/users/f69b7214196546e1ab79d2012a426a66/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:33.818445 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9248c1-022c-444e-9474-08b1a9214d39 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:33.818735 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9248c1-022c-444e-9474-08b1a9214d39 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:33.819013 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9248c1-022c-444e-9474-08b1a9214d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:33.845555 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-ed9248c1-022c-444e-9474-08b1a9214d39 None None] MFA Rules not processed for user `f69b7214196546e1ab79d2012a426a66`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:33.846752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ed9248c1-022c-444e-9474-08b1a9214d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:33.916728 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1232/2473] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:33 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:33.928066 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f775f-c027-4eab-886a-f36a1ebb28d7 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:33.929785 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c62f775f-c027-4eab-886a-f36a1ebb28d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:33.987538 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62f775f-c027-4eab-886a-f36a1ebb28d7 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:33.991558 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa5b85-f6ea-42f8-b1c6-0ce5a0fcdd17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:33.992159 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa5b85-f6ea-42f8-b1c6-0ce5a0fcdd17 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:33.992436 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa5b85-f6ea-42f8-b1c6-0ce5a0fcdd17 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:33.992704 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa5b85-f6ea-42f8-b1c6-0ce5a0fcdd17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:34.001476 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa5b85-f6ea-42f8-b1c6-0ce5a0fcdd17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:34.002948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa5b85-f6ea-42f8-b1c6-0ce5a0fcdd17 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:34.012849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1242/2474] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:48:33 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:48:34.060825 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9248c1-022c-444e-9474-08b1a9214d39 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:34.062209 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ed9248c1-022c-444e-9474-08b1a9214d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:34.081616 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9248c1-022c-444e-9474-08b1a9214d39 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:34.081616 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e7ecee-80f4-4686-819f-54aa92ea4270 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:34.081616 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7ecee-80f4-4686-819f-54aa92ea4270 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:34.081616 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7ecee-80f4-4686-819f-54aa92ea4270 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:34.082513 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7ecee-80f4-4686-819f-54aa92ea4270 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:34.085492 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:34.137772 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e7ecee-80f4-4686-819f-54aa92ea4270 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:34.139205 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e7ecee-80f4-4686-819f-54aa92ea4270 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:34.148609 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1233/2475] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:48:34 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:48:36.170662 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa5b85-f6ea-42f8-b1c6-0ce5a0fcdd17 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:36.180955 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa5b85-f6ea-42f8-b1c6-0ce5a0fcdd17 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:36.184543 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b668e-8fdd-49de-9898-9b9dd6b4bbde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:36.185129 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b668e-8fdd-49de-9898-9b9dd6b4bbde service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:36.185436 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b668e-8fdd-49de-9898-9b9dd6b4bbde service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:36.185700 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b668e-8fdd-49de-9898-9b9dd6b4bbde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:36.187431 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6b0b668e-8fdd-49de-9898-9b9dd6b4bbde service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:36.250991 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b668e-8fdd-49de-9898-9b9dd6b4bbde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:36.252425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b668e-8fdd-49de-9898-9b9dd6b4bbde service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:36.262765 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1243/2476] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:48:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:48:37.631885 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e7ecee-80f4-4686-819f-54aa92ea4270 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:37.633418 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e6e7ecee-80f4-4686-819f-54aa92ea4270 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:37.703080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e7ecee-80f4-4686-819f-54aa92ea4270 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:37.708133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ddf3d-24f0-4b15-9cac-4bac42e3d036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:37.708759 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ddf3d-24f0-4b15-9cac-4bac42e3d036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:37.709037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ddf3d-24f0-4b15-9cac-4bac42e3d036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:37.709341 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ddf3d-24f0-4b15-9cac-4bac42e3d036 admin admin] PATH_INFO: `/v3/users/7a669e81f3994fdea127294c43913bdc` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:37.710461 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ddf3d-24f0-4b15-9cac-4bac42e3d036 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a669e81f3994fdea127294c43913bdc)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:37.712409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ddf3d-24f0-4b15-9cac-4bac42e3d036 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:37.854341 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d86ddf3d-24f0-4b15-9cac-4bac42e3d036 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a669e81f3994fdea127294c43913bdc'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:37.855038 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d86ddf3d-24f0-4b15-9cac-4bac42e3d036 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a669e81f3994fdea127294c43913bdc'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:37.855629 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d86ddf3d-24f0-4b15-9cac-4bac42e3d036 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a669e81f3994fdea127294c43913bdc'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:37.863648 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d86ddf3d-24f0-4b15-9cac-4bac42e3d036 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a669e81f3994fdea127294c43913bdc'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:37.934656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d86ddf3d-24f0-4b15-9cac-4bac42e3d036 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a669e81f3994fdea127294c43913bdc'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:37.953387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1234/2477] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:48:37 2022] DELETE /identity/v3/users/7a669e81f3994fdea127294c43913bdc => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:37.958700 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57fefc-9f97-4257-93e0-0ac6fb21fbd8 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:37.959179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57fefc-9f97-4257-93e0-0ac6fb21fbd8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:37.959438 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57fefc-9f97-4257-93e0-0ac6fb21fbd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:37.978788 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-1d57fefc-9f97-4257-93e0-0ac6fb21fbd8 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:37.979975 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1d57fefc-9f97-4257-93e0-0ac6fb21fbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:37.983984 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:38.039721 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1244/2478] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:48:38.060721 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ddf3d-24f0-4b15-9cac-4bac42e3d036 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:38.062117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d86ddf3d-24f0-4b15-9cac-4bac42e3d036 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:38.095625 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:38.125164 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ddf3d-24f0-4b15-9cac-4bac42e3d036 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:38.129759 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621416f8-da08-4e1b-95ea-47e440ad0fcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:38.138879 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621416f8-da08-4e1b-95ea-47e440ad0fcb service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:38.138879 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621416f8-da08-4e1b-95ea-47e440ad0fcb service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:38.138879 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621416f8-da08-4e1b-95ea-47e440ad0fcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:38.140592 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621416f8-da08-4e1b-95ea-47e440ad0fcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:38.142083 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621416f8-da08-4e1b-95ea-47e440ad0fcb service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:38.153132 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1235/2479] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:48:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:48:38.582301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57fefc-9f97-4257-93e0-0ac6fb21fbd8 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:38.583902 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1d57fefc-9f97-4257-93e0-0ac6fb21fbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:38.598168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57fefc-9f97-4257-93e0-0ac6fb21fbd8 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:38.604176 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c39664-1c03-4603-9c53-c54999e4bf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:38.604825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c39664-1c03-4603-9c53-c54999e4bf0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:38.605118 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c39664-1c03-4603-9c53-c54999e4bf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:38.605456 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c39664-1c03-4603-9c53-c54999e4bf0c admin admin] PATH_INFO: `/v3/projects/27e36fc341ec4f369fa269b4585d8c9d` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:38.606519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c39664-1c03-4603-9c53-c54999e4bf0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=27e36fc341ec4f369fa269b4585d8c9d)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:38.608531 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c39664-1c03-4603-9c53-c54999e4bf0c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:38.728937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-96c39664-1c03-4603-9c53-c54999e4bf0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27e36fc341ec4f369fa269b4585d8c9d', 'request_id': 'req-96c39664-1c03-4603-9c53-c54999e4bf0c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:38.802717 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-96c39664-1c03-4603-9c53-c54999e4bf0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27e36fc341ec4f369fa269b4585d8c9d', 'request_id': 'req-96c39664-1c03-4603-9c53-c54999e4bf0c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:38.809240 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-96c39664-1c03-4603-9c53-c54999e4bf0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27e36fc341ec4f369fa269b4585d8c9d', 'request_id': 'req-96c39664-1c03-4603-9c53-c54999e4bf0c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:38.820479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-96c39664-1c03-4603-9c53-c54999e4bf0c admin admin] The token cache is being invalidate because project 27e36fc341ec4f369fa269b4585d8c9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:48:38.820763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-96c39664-1c03-4603-9c53-c54999e4bf0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:38.821407 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-96c39664-1c03-4603-9c53-c54999e4bf0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:38.822873 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1245/2480] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:48:38 2022] DELETE /identity/v3/projects/27e36fc341ec4f369fa269b4585d8c9d => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:38.827885 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70ebac-1e33-4ced-9559-61b09348248c None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:38.828339 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70ebac-1e33-4ced-9559-61b09348248c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:38.828452 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70ebac-1e33-4ced-9559-61b09348248c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:38.847888 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-7c70ebac-1e33-4ced-9559-61b09348248c None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:38.849055 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-7c70ebac-1e33-4ced-9559-61b09348248c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:38.906542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1236/2481] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:38.914394 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0be944-ec52-4fec-bc92-cb52d08b3c1a None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:38.914678 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0be944-ec52-4fec-bc92-cb52d08b3c1a None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:38.915062 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0be944-ec52-4fec-bc92-cb52d08b3c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:38.934340 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-ab0be944-ec52-4fec-bc92-cb52d08b3c1a None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:38.935529 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ab0be944-ec52-4fec-bc92-cb52d08b3c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:38.949357 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1246/2482] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:38.952297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70ebac-1e33-4ced-9559-61b09348248c None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:38.962868 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70ebac-1e33-4ced-9559-61b09348248c None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:38.967460 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a518ea79-fec0-45a6-a958-842712e6c0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:38.968074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518ea79-fec0-45a6-a958-842712e6c0a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:38.968351 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518ea79-fec0-45a6-a958-842712e6c0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:38.968617 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518ea79-fec0-45a6-a958-842712e6c0a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:38.969302 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a518ea79-fec0-45a6-a958-842712e6c0a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:48:38.969586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a518ea79-fec0-45a6-a958-842712e6c0a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:38.971142 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a518ea79-fec0-45a6-a958-842712e6c0a0 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:38.983741 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1237/2483] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:48:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:48:38.987916 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0be944-ec52-4fec-bc92-cb52d08b3c1a None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:38.999236 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0be944-ec52-4fec-bc92-cb52d08b3c1a None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:39.004640 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772e2ac-551d-4c9d-bce5-25b4c157f49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:39.005284 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772e2ac-551d-4c9d-bce5-25b4c157f49e admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:39.005639 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772e2ac-551d-4c9d-bce5-25b4c157f49e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:39.005922 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772e2ac-551d-4c9d-bce5-25b4c157f49e admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:39.006550 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2772e2ac-551d-4c9d-bce5-25b4c157f49e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:39.008449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2772e2ac-551d-4c9d-bce5-25b4c157f49e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:39.089938 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1247/2484] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:48:38 2022] POST /identity/v3/projects => generated 367 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:48:39.094125 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a518ea79-fec0-45a6-a958-842712e6c0a0 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:39.104156 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a518ea79-fec0-45a6-a958-842712e6c0a0 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:39.159733 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11d3a1-7b9b-4302-98cb-d1272d804b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:39.160424 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11d3a1-7b9b-4302-98cb-d1272d804b38 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:39.160705 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11d3a1-7b9b-4302-98cb-d1272d804b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:39.160976 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11d3a1-7b9b-4302-98cb-d1272d804b38 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:39.161637 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae11d3a1-7b9b-4302-98cb-d1272d804b38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:39.163527 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae11d3a1-7b9b-4302-98cb-d1272d804b38 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:39.237232 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1238/2485] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:48:39 2022] POST /identity/v3/users => generated 284 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:48:39.242002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772e2ac-551d-4c9d-bce5-25b4c157f49e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:39.255056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772e2ac-551d-4c9d-bce5-25b4c157f49e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:39.259736 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4029c-c285-4c02-b75f-f10aaeee2a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:39.260365 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4029c-c285-4c02-b75f-f10aaeee2a1d admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:39.260622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4029c-c285-4c02-b75f-f10aaeee2a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:39.260889 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4029c-c285-4c02-b75f-f10aaeee2a1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:39.261567 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4029c-c285-4c02-b75f-f10aaeee2a1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:39.263142 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4029c-c285-4c02-b75f-f10aaeee2a1d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:39.274134 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1248/2486] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:39.278129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11d3a1-7b9b-4302-98cb-d1272d804b38 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:39.288742 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11d3a1-7b9b-4302-98cb-d1272d804b38 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:39.293408 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9021de96-07f9-4757-84c9-e983587fd6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:39.294008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021de96-07f9-4757-84c9-e983587fd6ba admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:39.294289 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021de96-07f9-4757-84c9-e983587fd6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:39.294563 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021de96-07f9-4757-84c9-e983587fd6ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:39.295229 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9021de96-07f9-4757-84c9-e983587fd6ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:39.296886 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9021de96-07f9-4757-84c9-e983587fd6ba admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:39.308477 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1239/2487] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:39.312653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4029c-c285-4c02-b75f-f10aaeee2a1d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:39.323642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4029c-c285-4c02-b75f-f10aaeee2a1d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:39.328460 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530bea4-34d3-4c09-bba6-bf8903606cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:39.329104 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530bea4-34d3-4c09-bba6-bf8903606cbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:39.329428 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530bea4-34d3-4c09-bba6-bf8903606cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:39.329702 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530bea4-34d3-4c09-bba6-bf8903606cbd admin admin] PATH_INFO: `/v3/projects/752103b10916470eb60d43a9d0fb45a3/users/98638c29485a455eb6fb99f396ec552d/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:39.343966 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530bea4-34d3-4c09-bba6-bf8903606cbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=752103b10916470eb60d43a9d0fb45a3, user_id=98638c29485a455eb6fb99f396ec552d, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:39.346637 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530bea4-34d3-4c09-bba6-bf8903606cbd admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:39.477627 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1249/2488] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:48:39 2022] PUT /identity/v3/projects/752103b10916470eb60d43a9d0fb45a3/users/98638c29485a455eb6fb99f396ec552d/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:39.489633 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f156b57-4b94-40bc-9340-f5cea391bf05 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:39.489850 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f156b57-4b94-40bc-9340-f5cea391bf05 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:39.490109 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f156b57-4b94-40bc-9340-f5cea391bf05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:39.516557 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-0f156b57-4b94-40bc-9340-f5cea391bf05 None None] MFA Rules not processed for user `98638c29485a455eb6fb99f396ec552d`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:39.517774 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0f156b57-4b94-40bc-9340-f5cea391bf05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:39.540960 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:39.582292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1240/2489] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:39.592971 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530bea4-34d3-4c09-bba6-bf8903606cbd admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:39.594655 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b530bea4-34d3-4c09-bba6-bf8903606cbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:39.632346 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:39.660899 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530bea4-34d3-4c09-bba6-bf8903606cbd admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:39.664909 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b4926-a3e5-487e-a2f6-62d68ca89d8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:39.665612 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b4926-a3e5-487e-a2f6-62d68ca89d8d service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:39.665891 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b4926-a3e5-487e-a2f6-62d68ca89d8d service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:39.666155 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b4926-a3e5-487e-a2f6-62d68ca89d8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:39.676671 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081b4926-a3e5-487e-a2f6-62d68ca89d8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:39.678449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081b4926-a3e5-487e-a2f6-62d68ca89d8d service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:39.691651 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1250/2490] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:48:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:48:44.469444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6e791-fbc3-4d48-9a40-b2047d1d91e7 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:44.469914 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6e791-fbc3-4d48-9a40-b2047d1d91e7 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:44.470126 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6e791-fbc3-4d48-9a40-b2047d1d91e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:44.489720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-48c6e791-fbc3-4d48-9a40-b2047d1d91e7 None None] MFA Rules not processed for user `7f3422c6f9a041ceba851b71bd15d0df`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:44.490918 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-48c6e791-fbc3-4d48-9a40-b2047d1d91e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:44.547899 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1241/2491] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 08:48:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:44.704489 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b4926-a3e5-487e-a2f6-62d68ca89d8d service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:44.706820 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-081b4926-a3e5-487e-a2f6-62d68ca89d8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:44.782432 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b4926-a3e5-487e-a2f6-62d68ca89d8d service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:44.787732 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f06ed-81b6-4e91-ad4b-ad55afead7af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e69c0e02ad944f2a85fc5318018e5cb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:44.788539 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f06ed-81b6-4e91-ad4b-ad55afead7af service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:44.788821 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f06ed-81b6-4e91-ad4b-ad55afead7af service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:44.789576 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f06ed-81b6-4e91-ad4b-ad55afead7af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:44.798255 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897f06ed-81b6-4e91-ad4b-ad55afead7af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:44.799829 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897f06ed-81b6-4e91-ad4b-ad55afead7af service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:44.808489 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1251/2492] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 08:48:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 08:48:45.791162 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6e791-fbc3-4d48-9a40-b2047d1d91e7 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:45.792744 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-48c6e791-fbc3-4d48-9a40-b2047d1d91e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:45.864224 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6e791-fbc3-4d48-9a40-b2047d1d91e7 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:45.869391 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45626497-d17c-43b1-acbe-0c91f702ba20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:45.870188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45626497-d17c-43b1-acbe-0c91f702ba20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:45.870704 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45626497-d17c-43b1-acbe-0c91f702ba20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:45.871074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45626497-d17c-43b1-acbe-0c91f702ba20 admin admin] PATH_INFO: `/v3/users/377c22bb511141a7a09d8e1a2b9b0f2b` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:45.872318 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45626497-d17c-43b1-acbe-0c91f702ba20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=377c22bb511141a7a09d8e1a2b9b0f2b)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:45.874476 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45626497-d17c-43b1-acbe-0c91f702ba20 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:46.048114 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-45626497-d17c-43b1-acbe-0c91f702ba20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '377c22bb511141a7a09d8e1a2b9b0f2b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:46.049111 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-45626497-d17c-43b1-acbe-0c91f702ba20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '377c22bb511141a7a09d8e1a2b9b0f2b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:46.049966 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-45626497-d17c-43b1-acbe-0c91f702ba20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '377c22bb511141a7a09d8e1a2b9b0f2b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:46.062869 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-45626497-d17c-43b1-acbe-0c91f702ba20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '377c22bb511141a7a09d8e1a2b9b0f2b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:46.128575 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-45626497-d17c-43b1-acbe-0c91f702ba20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '377c22bb511141a7a09d8e1a2b9b0f2b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:46.139683 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:46.152717 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1242/2493] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:48:45 2022] DELETE /identity/v3/users/377c22bb511141a7a09d8e1a2b9b0f2b => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:48:46.157028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024cf71-3c82-4fba-83bb-88495f8201fb None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:46.157534 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024cf71-3c82-4fba-83bb-88495f8201fb None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:46.157647 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024cf71-3c82-4fba-83bb-88495f8201fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:46.181406 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-7024cf71-3c82-4fba-83bb-88495f8201fb None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:46.182737 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7024cf71-3c82-4fba-83bb-88495f8201fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:46.258579 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1252/2494] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:46.274846 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45626497-d17c-43b1-acbe-0c91f702ba20 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:46.276596 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-45626497-d17c-43b1-acbe-0c91f702ba20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:46.364368 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45626497-d17c-43b1-acbe-0c91f702ba20 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:46.368605 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8bf37-94be-4004-a49a-6dc75af1bfe6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:46.369617 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8bf37-94be-4004-a49a-6dc75af1bfe6 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:46.369867 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8bf37-94be-4004-a49a-6dc75af1bfe6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:46.370204 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8bf37-94be-4004-a49a-6dc75af1bfe6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:46.381586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea8bf37-94be-4004-a49a-6dc75af1bfe6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:46.383698 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea8bf37-94be-4004-a49a-6dc75af1bfe6 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:46.415525 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1243/2495] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:48:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:48:46.866416 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024cf71-3c82-4fba-83bb-88495f8201fb None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:46.869241 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7024cf71-3c82-4fba-83bb-88495f8201fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:46.911036 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024cf71-3c82-4fba-83bb-88495f8201fb None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:46.924269 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ef965-d33c-4dab-b56f-56bdc8ff583a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:46.925264 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ef965-d33c-4dab-b56f-56bdc8ff583a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:46.925745 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ef965-d33c-4dab-b56f-56bdc8ff583a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:46.926174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ef965-d33c-4dab-b56f-56bdc8ff583a admin admin] PATH_INFO: `/v3/projects/833ae0a6ab4e4990ac713489b3a2a8f4` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:46.927964 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ef965-d33c-4dab-b56f-56bdc8ff583a admin admin] RBAC: Authorizing `identity:delete_project(project_id=833ae0a6ab4e4990ac713489b3a2a8f4)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:46.930681 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ef965-d33c-4dab-b56f-56bdc8ff583a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:47.139810 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bf8ef965-d33c-4dab-b56f-56bdc8ff583a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '833ae0a6ab4e4990ac713489b3a2a8f4', 'request_id': 'req-bf8ef965-d33c-4dab-b56f-56bdc8ff583a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:47.202951 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bf8ef965-d33c-4dab-b56f-56bdc8ff583a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '833ae0a6ab4e4990ac713489b3a2a8f4', 'request_id': 'req-bf8ef965-d33c-4dab-b56f-56bdc8ff583a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:47.204520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:47.212664 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bf8ef965-d33c-4dab-b56f-56bdc8ff583a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '833ae0a6ab4e4990ac713489b3a2a8f4', 'request_id': 'req-bf8ef965-d33c-4dab-b56f-56bdc8ff583a'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:47.232758 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bf8ef965-d33c-4dab-b56f-56bdc8ff583a admin admin] The token cache is being invalidate because project 833ae0a6ab4e4990ac713489b3a2a8f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:48:47.233022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bf8ef965-d33c-4dab-b56f-56bdc8ff583a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:47.233736 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bf8ef965-d33c-4dab-b56f-56bdc8ff583a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:47.235641 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1253/2496] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:48:46 2022] DELETE /identity/v3/projects/833ae0a6ab4e4990ac713489b3a2a8f4 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:47.240206 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8bf37-94be-4004-a49a-6dc75af1bfe6 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:47.252225 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8bf37-94be-4004-a49a-6dc75af1bfe6 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:47.347228 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73d2fa-25a8-4886-a121-84606a308e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:47.348633 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73d2fa-25a8-4886-a121-84606a308e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:47.349167 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73d2fa-25a8-4886-a121-84606a308e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:47.349502 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73d2fa-25a8-4886-a121-84606a308e37 admin admin] PATH_INFO: `/v3/users/f69b7214196546e1ab79d2012a426a66` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:47.351231 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73d2fa-25a8-4886-a121-84606a308e37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f69b7214196546e1ab79d2012a426a66)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:47.355447 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73d2fa-25a8-4886-a121-84606a308e37 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:47.544942 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4e73d2fa-25a8-4886-a121-84606a308e37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f69b7214196546e1ab79d2012a426a66'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:47.545617 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4e73d2fa-25a8-4886-a121-84606a308e37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f69b7214196546e1ab79d2012a426a66'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:47.546215 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4e73d2fa-25a8-4886-a121-84606a308e37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f69b7214196546e1ab79d2012a426a66'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:47.553905 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4e73d2fa-25a8-4886-a121-84606a308e37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f69b7214196546e1ab79d2012a426a66'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:47.843550 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-4e73d2fa-25a8-4886-a121-84606a308e37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f69b7214196546e1ab79d2012a426a66'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:47.861963 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1244/2497] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:48:47 2022] DELETE /identity/v3/users/f69b7214196546e1ab79d2012a426a66 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:48:48.262197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ef965-d33c-4dab-b56f-56bdc8ff583a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:48.263728 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bf8ef965-d33c-4dab-b56f-56bdc8ff583a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:48.278872 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ef965-d33c-4dab-b56f-56bdc8ff583a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:48.336122 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64162088-72de-4740-8c57-c3684a63d8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:48.336779 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64162088-72de-4740-8c57-c3684a63d8e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:48.337059 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64162088-72de-4740-8c57-c3684a63d8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:48.337360 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64162088-72de-4740-8c57-c3684a63d8e6 admin admin] PATH_INFO: `/v3/projects/a2f4c25ce1c34cb4bbde376be503d54d` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:48.338518 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64162088-72de-4740-8c57-c3684a63d8e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2f4c25ce1c34cb4bbde376be503d54d)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:48.340442 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64162088-72de-4740-8c57-c3684a63d8e6 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:48.426990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-64162088-72de-4740-8c57-c3684a63d8e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2f4c25ce1c34cb4bbde376be503d54d', 'request_id': 'req-64162088-72de-4740-8c57-c3684a63d8e6'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:48.487690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-64162088-72de-4740-8c57-c3684a63d8e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2f4c25ce1c34cb4bbde376be503d54d', 'request_id': 'req-64162088-72de-4740-8c57-c3684a63d8e6'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:48.494774 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-64162088-72de-4740-8c57-c3684a63d8e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2f4c25ce1c34cb4bbde376be503d54d', 'request_id': 'req-64162088-72de-4740-8c57-c3684a63d8e6'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:48.505672 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-64162088-72de-4740-8c57-c3684a63d8e6 admin admin] The token cache is being invalidate because project a2f4c25ce1c34cb4bbde376be503d54d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:48:48.505897 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-64162088-72de-4740-8c57-c3684a63d8e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:48.506522 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-64162088-72de-4740-8c57-c3684a63d8e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:48.508689 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1254/2498] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:48:48 2022] DELETE /identity/v3/projects/a2f4c25ce1c34cb4bbde376be503d54d => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:48.513691 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443d76c-37e0-4fb8-976d-e33ddf960ff9 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:48.514153 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443d76c-37e0-4fb8-976d-e33ddf960ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:48.514251 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443d76c-37e0-4fb8-976d-e33ddf960ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:48.534000 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-a443d76c-37e0-4fb8-976d-e33ddf960ff9 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:48.535190 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a443d76c-37e0-4fb8-976d-e33ddf960ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:48.540690 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:48.593277 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1245/2499] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:48.602518 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e7929-7442-4fb5-ae40-8513c7c680b6 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:48.602830 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e7929-7442-4fb5-ae40-8513c7c680b6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:48.603114 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e7929-7442-4fb5-ae40-8513c7c680b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:48.626434 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-f18e7929-7442-4fb5-ae40-8513c7c680b6 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:48.627724 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f18e7929-7442-4fb5-ae40-8513c7c680b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:48.636573 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1255/2500] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:48:48.640755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443d76c-37e0-4fb8-976d-e33ddf960ff9 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:48.652227 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443d76c-37e0-4fb8-976d-e33ddf960ff9 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:48.657767 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e1370-e3de-4829-b25c-4fa2a6768deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:48.658366 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e1370-e3de-4829-b25c-4fa2a6768deb admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:48.658620 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e1370-e3de-4829-b25c-4fa2a6768deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:48.658718 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e1370-e3de-4829-b25c-4fa2a6768deb admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:48.660537 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e1370-e3de-4829-b25c-4fa2a6768deb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:48:48.661104 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e1370-e3de-4829-b25c-4fa2a6768deb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:48.663697 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e1370-e3de-4829-b25c-4fa2a6768deb admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:48.682941 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1246/2501] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:48:48.683825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18e7929-7442-4fb5-ae40-8513c7c680b6 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:48.694403 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18e7929-7442-4fb5-ae40-8513c7c680b6 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:48.703693 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c3ad6-9aae-4d2d-90f7-eddf809fd039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:48.705447 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c3ad6-9aae-4d2d-90f7-eddf809fd039 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:48.705748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c3ad6-9aae-4d2d-90f7-eddf809fd039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:48.706400 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c3ad6-9aae-4d2d-90f7-eddf809fd039 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:48.707414 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70c3ad6-9aae-4d2d-90f7-eddf809fd039 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:48.709991 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70c3ad6-9aae-4d2d-90f7-eddf809fd039 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:48.805893 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1256/2502] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:48:48 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:48:48.810687 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e1370-e3de-4829-b25c-4fa2a6768deb admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:48.827819 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e1370-e3de-4829-b25c-4fa2a6768deb admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:48.892056 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea75c9-708d-447d-9d51-47cf6c4c39da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:48.892828 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea75c9-708d-447d-9d51-47cf6c4c39da admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:48.893246 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea75c9-708d-447d-9d51-47cf6c4c39da admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:48.893649 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea75c9-708d-447d-9d51-47cf6c4c39da admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:48.894374 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea75c9-708d-447d-9d51-47cf6c4c39da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:48.896396 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea75c9-708d-447d-9d51-47cf6c4c39da admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:48.988641 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1247/2503] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:48:48 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:48:48.993924 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c3ad6-9aae-4d2d-90f7-eddf809fd039 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:49.009695 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c3ad6-9aae-4d2d-90f7-eddf809fd039 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:49.030008 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052514a-8ba5-4b58-a180-31767379ce6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:49.031128 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052514a-8ba5-4b58-a180-31767379ce6c admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:49.031578 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052514a-8ba5-4b58-a180-31767379ce6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:49.032158 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052514a-8ba5-4b58-a180-31767379ce6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:49.033067 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7052514a-8ba5-4b58-a180-31767379ce6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:49.035483 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7052514a-8ba5-4b58-a180-31767379ce6c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:49.050230 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1257/2504] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:48:49.053958 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea75c9-708d-447d-9d51-47cf6c4c39da admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:49.066223 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea75c9-708d-447d-9d51-47cf6c4c39da admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:49.076246 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f020d-0755-4ca0-980f-d4c68f594a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:49.077112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f020d-0755-4ca0-980f-d4c68f594a3d admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:49.077635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f020d-0755-4ca0-980f-d4c68f594a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:49.077765 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f020d-0755-4ca0-980f-d4c68f594a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:49.079308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f020d-0755-4ca0-980f-d4c68f594a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:49.082757 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f020d-0755-4ca0-980f-d4c68f594a3d admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:49.095776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1248/2505] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:49.099655 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052514a-8ba5-4b58-a180-31767379ce6c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:49.111162 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052514a-8ba5-4b58-a180-31767379ce6c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:49.133071 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3154b29-b5f1-4d64-ae42-8432be973aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:49.134199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3154b29-b5f1-4d64-ae42-8432be973aa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:49.134706 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3154b29-b5f1-4d64-ae42-8432be973aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:49.139475 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3154b29-b5f1-4d64-ae42-8432be973aa4 admin admin] PATH_INFO: `/v3/projects/1dd996823ee6481bb037f2072fae7a68/users/5dad9aa2313b4af6b646a994d35c2fb2/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:49.166382 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3154b29-b5f1-4d64-ae42-8432be973aa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dd996823ee6481bb037f2072fae7a68, user_id=5dad9aa2313b4af6b646a994d35c2fb2, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:49.174386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3154b29-b5f1-4d64-ae42-8432be973aa4 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:49.261219 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1258/2506] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:48:49 2022] PUT /identity/v3/projects/1dd996823ee6481bb037f2072fae7a68/users/5dad9aa2313b4af6b646a994d35c2fb2/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:49.271270 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e1325-51a2-437f-932f-f36612421a2c None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:49.271604 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e1325-51a2-437f-932f-f36612421a2c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:49.271880 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e1325-51a2-437f-932f-f36612421a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:49.323903 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-735e1325-51a2-437f-932f-f36612421a2c None None] MFA Rules not processed for user `5dad9aa2313b4af6b646a994d35c2fb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:49.325229 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-735e1325-51a2-437f-932f-f36612421a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:49.408932 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1249/2507] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:49.414753 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3154b29-b5f1-4d64-ae42-8432be973aa4 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:49.439029 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:49.563307 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3154b29-b5f1-4d64-ae42-8432be973aa4 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:49.568430 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f0fdf-07e0-46b0-8454-5de9364752dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:49.569214 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f0fdf-07e0-46b0-8454-5de9364752dc admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:49.569567 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f0fdf-07e0-46b0-8454-5de9364752dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:49.570543 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f0fdf-07e0-46b0-8454-5de9364752dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:49.570733 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f0fdf-07e0-46b0-8454-5de9364752dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:49.573040 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f0fdf-07e0-46b0-8454-5de9364752dc admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:49.715985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1259/2508] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:48:49 2022] POST /identity/v3/projects => generated 385 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:48:49.721723 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e1325-51a2-437f-932f-f36612421a2c None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:49.741522 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e1325-51a2-437f-932f-f36612421a2c None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:49.753756 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:49.868327 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e05d74-feec-432d-af14-d2f94de8cce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:49.868327 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e05d74-feec-432d-af14-d2f94de8cce2 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:49.868327 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e05d74-feec-432d-af14-d2f94de8cce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:49.868327 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e05d74-feec-432d-af14-d2f94de8cce2 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:49.868735 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e05d74-feec-432d-af14-d2f94de8cce2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:49.870703 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e05d74-feec-432d-af14-d2f94de8cce2 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:49.952545 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1250/2509] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:48:49 2022] POST /identity/v3/users => generated 299 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:48:49.956900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f0fdf-07e0-46b0-8454-5de9364752dc admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:49.970299 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f0fdf-07e0-46b0-8454-5de9364752dc admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:49.975409 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81a76e-89f4-4149-9c3a-a062b2357c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:49.976401 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81a76e-89f4-4149-9c3a-a062b2357c78 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:49.976832 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81a76e-89f4-4149-9c3a-a062b2357c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:49.977298 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81a76e-89f4-4149-9c3a-a062b2357c78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:49.977936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d81a76e-89f4-4149-9c3a-a062b2357c78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:49.989277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d81a76e-89f4-4149-9c3a-a062b2357c78 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:50.001383 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1260/2510] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:50.002221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e05d74-feec-432d-af14-d2f94de8cce2 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:50.021117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e05d74-feec-432d-af14-d2f94de8cce2 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:50.027257 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c027edb-4eb2-4ee4-94ff-ccb148f774c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:50.028134 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c027edb-4eb2-4ee4-94ff-ccb148f774c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:50.028509 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c027edb-4eb2-4ee4-94ff-ccb148f774c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:50.028967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c027edb-4eb2-4ee4-94ff-ccb148f774c8 admin admin] PATH_INFO: `/v3/projects/d371551d7dac4170b1aaf19f7e6b7056/users/e9a475b702354487a6a1493b85b6a73c/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:50.048044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c027edb-4eb2-4ee4-94ff-ccb148f774c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d371551d7dac4170b1aaf19f7e6b7056, user_id=e9a475b702354487a6a1493b85b6a73c, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:50.053498 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c027edb-4eb2-4ee4-94ff-ccb148f774c8 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:50.192210 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1251/2511] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:48:50 2022] PUT /identity/v3/projects/d371551d7dac4170b1aaf19f7e6b7056/users/e9a475b702354487a6a1493b85b6a73c/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:48:50.193400 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81a76e-89f4-4149-9c3a-a062b2357c78 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:50.204511 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81a76e-89f4-4149-9c3a-a062b2357c78 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:50.284399 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8d2a6d-e12c-4e09-9386-0cdaa08faea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:50.285030 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d2a6d-e12c-4e09-9386-0cdaa08faea4 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:50.285356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d2a6d-e12c-4e09-9386-0cdaa08faea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:50.285671 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d2a6d-e12c-4e09-9386-0cdaa08faea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:50.286346 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8d2a6d-e12c-4e09-9386-0cdaa08faea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:50.288071 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8d2a6d-e12c-4e09-9386-0cdaa08faea4 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:50.301185 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1261/2512] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:50.305839 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c027edb-4eb2-4ee4-94ff-ccb148f774c8 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:50.325988 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c027edb-4eb2-4ee4-94ff-ccb148f774c8 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:50.332010 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a93671-8443-4e76-9336-66e99a4dc087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:50.332772 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a93671-8443-4e76-9336-66e99a4dc087 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:50.333117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a93671-8443-4e76-9336-66e99a4dc087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:50.333449 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a93671-8443-4e76-9336-66e99a4dc087 admin admin] PATH_INFO: `/v3/projects/d371551d7dac4170b1aaf19f7e6b7056/users/e9a475b702354487a6a1493b85b6a73c/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:50.340739 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a93671-8443-4e76-9336-66e99a4dc087 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d371551d7dac4170b1aaf19f7e6b7056, user_id=e9a475b702354487a6a1493b85b6a73c, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:50.348928 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a93671-8443-4e76-9336-66e99a4dc087 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:50.373711 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1252/2513] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:48:50 2022] PUT /identity/v3/projects/d371551d7dac4170b1aaf19f7e6b7056/users/e9a475b702354487a6a1493b85b6a73c/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:50.385062 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326f589-1cbf-4016-b4b7-144e5afba0d4 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:50.386227 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326f589-1cbf-4016-b4b7-144e5afba0d4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:50.387477 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326f589-1cbf-4016-b4b7-144e5afba0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:50.457297 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e326f589-1cbf-4016-b4b7-144e5afba0d4 None None] MFA Rules not processed for user `e9a475b702354487a6a1493b85b6a73c`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:50.458688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e326f589-1cbf-4016-b4b7-144e5afba0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:50.547656 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1262/2514] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:50 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:50.552457 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a93671-8443-4e76-9336-66e99a4dc087 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:50.561858 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f3a93671-8443-4e76-9336-66e99a4dc087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:50.635966 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a93671-8443-4e76-9336-66e99a4dc087 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:50.639640 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b23d77-17f2-4352-81c4-fcc0abaf6503 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:50.640402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b23d77-17f2-4352-81c4-fcc0abaf6503 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:50.640600 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b23d77-17f2-4352-81c4-fcc0abaf6503 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:50.640992 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b23d77-17f2-4352-81c4-fcc0abaf6503 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:50.652626 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b23d77-17f2-4352-81c4-fcc0abaf6503 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:50.654349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b23d77-17f2-4352-81c4-fcc0abaf6503 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:50.666387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1253/2515] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:48:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:48:52.300746 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326f589-1cbf-4016-b4b7-144e5afba0d4 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:52.310642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:52.373527 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326f589-1cbf-4016-b4b7-144e5afba0d4 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:52.378927 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a758e1-8df5-459b-b2d5-ef71d095861e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:52.379733 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a758e1-8df5-459b-b2d5-ef71d095861e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:52.380072 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a758e1-8df5-459b-b2d5-ef71d095861e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:52.380722 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a758e1-8df5-459b-b2d5-ef71d095861e admin admin] PATH_INFO: `/v3/users/5dad9aa2313b4af6b646a994d35c2fb2` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:52.381922 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a758e1-8df5-459b-b2d5-ef71d095861e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dad9aa2313b4af6b646a994d35c2fb2)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:52.384601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a758e1-8df5-459b-b2d5-ef71d095861e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:52.554743 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-53a758e1-8df5-459b-b2d5-ef71d095861e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dad9aa2313b4af6b646a994d35c2fb2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:52.555514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-53a758e1-8df5-459b-b2d5-ef71d095861e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dad9aa2313b4af6b646a994d35c2fb2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:52.556200 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-53a758e1-8df5-459b-b2d5-ef71d095861e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dad9aa2313b4af6b646a994d35c2fb2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:52.563286 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-53a758e1-8df5-459b-b2d5-ef71d095861e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dad9aa2313b4af6b646a994d35c2fb2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:52.659299 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-53a758e1-8df5-459b-b2d5-ef71d095861e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dad9aa2313b4af6b646a994d35c2fb2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:52.677728 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1263/2516] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:48:52 2022] DELETE /identity/v3/users/5dad9aa2313b4af6b646a994d35c2fb2 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:52.682771 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46dcc2c-8c20-4981-a409-92856a73e582 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:52.683251 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46dcc2c-8c20-4981-a409-92856a73e582 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:52.683350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46dcc2c-8c20-4981-a409-92856a73e582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:52.703489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-e46dcc2c-8c20-4981-a409-92856a73e582 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:52.704758 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e46dcc2c-8c20-4981-a409-92856a73e582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:52.761900 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1254/2517] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:52.771362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a758e1-8df5-459b-b2d5-ef71d095861e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:52.772859 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-53a758e1-8df5-459b-b2d5-ef71d095861e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:52.831294 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a758e1-8df5-459b-b2d5-ef71d095861e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:52.836807 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7be9b2-9be2-4d82-9118-74cb90eb261b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:52.837455 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7be9b2-9be2-4d82-9118-74cb90eb261b service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:52.837734 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7be9b2-9be2-4d82-9118-74cb90eb261b service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:52.838004 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7be9b2-9be2-4d82-9118-74cb90eb261b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:52.847166 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7be9b2-9be2-4d82-9118-74cb90eb261b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:52.848610 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7be9b2-9be2-4d82-9118-74cb90eb261b service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:52.858800 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1264/2518] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:48:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:48:53.233819 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46dcc2c-8c20-4981-a409-92856a73e582 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:53.236359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e46dcc2c-8c20-4981-a409-92856a73e582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:53.242845 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:53.248275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46dcc2c-8c20-4981-a409-92856a73e582 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:53.252779 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5386f1ce-73dd-4070-a799-48bbeda682ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:53.253454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386f1ce-73dd-4070-a799-48bbeda682ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:53.253719 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386f1ce-73dd-4070-a799-48bbeda682ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:53.253989 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386f1ce-73dd-4070-a799-48bbeda682ed admin admin] PATH_INFO: `/v3/projects/1dd996823ee6481bb037f2072fae7a68` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:53.255085 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5386f1ce-73dd-4070-a799-48bbeda682ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dd996823ee6481bb037f2072fae7a68)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:53.257049 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5386f1ce-73dd-4070-a799-48bbeda682ed admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:53.369259 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5386f1ce-73dd-4070-a799-48bbeda682ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dd996823ee6481bb037f2072fae7a68', 'request_id': 'req-5386f1ce-73dd-4070-a799-48bbeda682ed'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:53.382055 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5386f1ce-73dd-4070-a799-48bbeda682ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dd996823ee6481bb037f2072fae7a68', 'request_id': 'req-5386f1ce-73dd-4070-a799-48bbeda682ed'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:53.382055 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5386f1ce-73dd-4070-a799-48bbeda682ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dd996823ee6481bb037f2072fae7a68', 'request_id': 'req-5386f1ce-73dd-4070-a799-48bbeda682ed'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:53.441741 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5386f1ce-73dd-4070-a799-48bbeda682ed admin admin] The token cache is being invalidate because project 1dd996823ee6481bb037f2072fae7a68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:48:53.442153 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5386f1ce-73dd-4070-a799-48bbeda682ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:53.442760 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-5386f1ce-73dd-4070-a799-48bbeda682ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:53.444323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1255/2519] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:48:53 2022] DELETE /identity/v3/projects/1dd996823ee6481bb037f2072fae7a68 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:48:53.448243 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7be9b2-9be2-4d82-9118-74cb90eb261b service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:53.457971 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7be9b2-9be2-4d82-9118-74cb90eb261b service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:53.484971 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:53.514426 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e578566-4d72-4194-a852-6ca54ac6dcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:53.515081 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e578566-4d72-4194-a852-6ca54ac6dcb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:53.515585 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e578566-4d72-4194-a852-6ca54ac6dcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:53.516042 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e578566-4d72-4194-a852-6ca54ac6dcb0 admin admin] PATH_INFO: `/v3/users/e9a475b702354487a6a1493b85b6a73c` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:53.517546 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e578566-4d72-4194-a852-6ca54ac6dcb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9a475b702354487a6a1493b85b6a73c)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:53.519484 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e578566-4d72-4194-a852-6ca54ac6dcb0 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:53.750597 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9e578566-4d72-4194-a852-6ca54ac6dcb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9a475b702354487a6a1493b85b6a73c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:53.751324 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9e578566-4d72-4194-a852-6ca54ac6dcb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9a475b702354487a6a1493b85b6a73c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:53.752008 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9e578566-4d72-4194-a852-6ca54ac6dcb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9a475b702354487a6a1493b85b6a73c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:53.760667 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9e578566-4d72-4194-a852-6ca54ac6dcb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9a475b702354487a6a1493b85b6a73c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:53.815022 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9e578566-4d72-4194-a852-6ca54ac6dcb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9a475b702354487a6a1493b85b6a73c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:53.835693 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1265/2520] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:48:53 2022] DELETE /identity/v3/users/e9a475b702354487a6a1493b85b6a73c => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:54.221603 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5386f1ce-73dd-4070-a799-48bbeda682ed admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:54.223348 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5386f1ce-73dd-4070-a799-48bbeda682ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:54.248532 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5386f1ce-73dd-4070-a799-48bbeda682ed admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:54.330484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daf70cd-8f1c-4d95-b942-8263db888048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:54.331371 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf70cd-8f1c-4d95-b942-8263db888048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:54.331695 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf70cd-8f1c-4d95-b942-8263db888048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:54.332031 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf70cd-8f1c-4d95-b942-8263db888048 admin admin] PATH_INFO: `/v3/projects/d371551d7dac4170b1aaf19f7e6b7056` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:54.333324 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daf70cd-8f1c-4d95-b942-8263db888048 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d371551d7dac4170b1aaf19f7e6b7056)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:54.335285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daf70cd-8f1c-4d95-b942-8263db888048 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:54.443391 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-7daf70cd-8f1c-4d95-b942-8263db888048 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd371551d7dac4170b1aaf19f7e6b7056', 'request_id': 'req-7daf70cd-8f1c-4d95-b942-8263db888048'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:54.451785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-7daf70cd-8f1c-4d95-b942-8263db888048 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd371551d7dac4170b1aaf19f7e6b7056', 'request_id': 'req-7daf70cd-8f1c-4d95-b942-8263db888048'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:54.458043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-7daf70cd-8f1c-4d95-b942-8263db888048 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd371551d7dac4170b1aaf19f7e6b7056', 'request_id': 'req-7daf70cd-8f1c-4d95-b942-8263db888048'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:54.505692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-7daf70cd-8f1c-4d95-b942-8263db888048 admin admin] The token cache is being invalidate because project d371551d7dac4170b1aaf19f7e6b7056 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:48:54.506127 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-7daf70cd-8f1c-4d95-b942-8263db888048 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:54.507005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-7daf70cd-8f1c-4d95-b942-8263db888048 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:54.508911 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1256/2521] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:48:54 2022] DELETE /identity/v3/projects/d371551d7dac4170b1aaf19f7e6b7056 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:54.513640 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319e47c-72d5-4104-8907-d8ebf1e2ec07 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:54.514175 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319e47c-72d5-4104-8907-d8ebf1e2ec07 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:54.514392 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319e47c-72d5-4104-8907-d8ebf1e2ec07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:54.534671 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-1319e47c-72d5-4104-8907-d8ebf1e2ec07 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:54.536234 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1319e47c-72d5-4104-8907-d8ebf1e2ec07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:54.600187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1266/2522] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:54.609871 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5925d2-f379-42fa-bfa7-fbbcbd0351e0 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:54.610159 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5925d2-f379-42fa-bfa7-fbbcbd0351e0 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:54.610435 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5925d2-f379-42fa-bfa7-fbbcbd0351e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:54.634824 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-7b5925d2-f379-42fa-bfa7-fbbcbd0351e0 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:54.636061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-7b5925d2-f379-42fa-bfa7-fbbcbd0351e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:54.650383 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319e47c-72d5-4104-8907-d8ebf1e2ec07 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:54.650914 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1257/2523] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:54.660179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319e47c-72d5-4104-8907-d8ebf1e2ec07 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:54.666050 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183bee60-5aa5-410b-b9ee-012565ccdba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:54.667266 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bee60-5aa5-410b-b9ee-012565ccdba3 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:54.667602 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bee60-5aa5-410b-b9ee-012565ccdba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:54.667823 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bee60-5aa5-410b-b9ee-012565ccdba3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:54.668480 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183bee60-5aa5-410b-b9ee-012565ccdba3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:48:54.668884 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183bee60-5aa5-410b-b9ee-012565ccdba3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:54.670530 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183bee60-5aa5-410b-b9ee-012565ccdba3 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:54.684398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1267/2524] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:48:54.688837 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5925d2-f379-42fa-bfa7-fbbcbd0351e0 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:54.702173 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5925d2-f379-42fa-bfa7-fbbcbd0351e0 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:54.712603 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890fb78-a0a2-4614-98ea-4f127b95bf63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:54.713537 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890fb78-a0a2-4614-98ea-4f127b95bf63 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:54.713946 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890fb78-a0a2-4614-98ea-4f127b95bf63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:54.714416 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890fb78-a0a2-4614-98ea-4f127b95bf63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:54.715228 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890fb78-a0a2-4614-98ea-4f127b95bf63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:54.718088 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890fb78-a0a2-4614-98ea-4f127b95bf63 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:54.725955 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:54.819451 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1258/2525] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:48:54 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:48:54.820590 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183bee60-5aa5-410b-b9ee-012565ccdba3 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:54.832059 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183bee60-5aa5-410b-b9ee-012565ccdba3 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:54.874158 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:54.903571 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a8766-da4d-48b8-9b10-253480c4f4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:54.904650 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a8766-da4d-48b8-9b10-253480c4f4f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:54.904995 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a8766-da4d-48b8-9b10-253480c4f4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:54.905340 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a8766-da4d-48b8-9b10-253480c4f4f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:54.905970 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a8766-da4d-48b8-9b10-253480c4f4f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:54.907937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a8766-da4d-48b8-9b10-253480c4f4f3 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:55.016198 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1268/2526] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:48:54 2022] POST /identity/v3/users => generated 292 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:48:55.020191 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890fb78-a0a2-4614-98ea-4f127b95bf63 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:55.033269 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890fb78-a0a2-4614-98ea-4f127b95bf63 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:55.038550 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5e6ab-f8de-45e7-91e9-543e41477c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:55.040283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5e6ab-f8de-45e7-91e9-543e41477c75 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:55.040619 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5e6ab-f8de-45e7-91e9-543e41477c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:55.041049 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5e6ab-f8de-45e7-91e9-543e41477c75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:55.041720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5e6ab-f8de-45e7-91e9-543e41477c75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:55.043326 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5e6ab-f8de-45e7-91e9-543e41477c75 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:55.055553 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1259/2527] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:48:55.059448 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a8766-da4d-48b8-9b10-253480c4f4f3 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:55.075067 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a8766-da4d-48b8-9b10-253480c4f4f3 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:55.081420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35cbb30-63fc-469b-8c4e-83f26f131b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:55.082120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cbb30-63fc-469b-8c4e-83f26f131b4b admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:55.082494 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cbb30-63fc-469b-8c4e-83f26f131b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:55.082834 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cbb30-63fc-469b-8c4e-83f26f131b4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:55.083856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35cbb30-63fc-469b-8c4e-83f26f131b4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:55.086508 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35cbb30-63fc-469b-8c4e-83f26f131b4b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:55.105759 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1269/2528] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:55.109683 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5e6ab-f8de-45e7-91e9-543e41477c75 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:55.130265 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5e6ab-f8de-45e7-91e9-543e41477c75 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:55.135145 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12963fe9-849c-4c71-b7ff-58a50b5a78c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:55.135865 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12963fe9-849c-4c71-b7ff-58a50b5a78c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:55.136155 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12963fe9-849c-4c71-b7ff-58a50b5a78c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:55.136426 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12963fe9-849c-4c71-b7ff-58a50b5a78c4 admin admin] PATH_INFO: `/v3/projects/3597de62832549adab0206d0ba797e74/users/b1af67c1a77e42cdb0652eb39b2a7e57/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:55.153960 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12963fe9-849c-4c71-b7ff-58a50b5a78c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3597de62832549adab0206d0ba797e74, user_id=b1af67c1a77e42cdb0652eb39b2a7e57, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:55.156691 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12963fe9-849c-4c71-b7ff-58a50b5a78c4 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:55.232378 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1260/2529] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:48:55 2022] PUT /identity/v3/projects/3597de62832549adab0206d0ba797e74/users/b1af67c1a77e42cdb0652eb39b2a7e57/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:55.245626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39d47d-3821-4c10-bd30-8fc21615b299 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:55.246108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39d47d-3821-4c10-bd30-8fc21615b299 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:55.246545 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39d47d-3821-4c10-bd30-8fc21615b299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:55.287964 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-bb39d47d-3821-4c10-bd30-8fc21615b299 None None] MFA Rules not processed for user `b1af67c1a77e42cdb0652eb39b2a7e57`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:55.290535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bb39d47d-3821-4c10-bd30-8fc21615b299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:55.372282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1270/2530] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:55.376621 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12963fe9-849c-4c71-b7ff-58a50b5a78c4 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:55.454793 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12963fe9-849c-4c71-b7ff-58a50b5a78c4 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:55.460118 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cafcdc8-db02-4f8e-9c22-d2f30bb4bc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:55.461860 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cafcdc8-db02-4f8e-9c22-d2f30bb4bc99 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:55.462294 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cafcdc8-db02-4f8e-9c22-d2f30bb4bc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:55.462735 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cafcdc8-db02-4f8e-9c22-d2f30bb4bc99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:55.463378 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cafcdc8-db02-4f8e-9c22-d2f30bb4bc99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:55.465306 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cafcdc8-db02-4f8e-9c22-d2f30bb4bc99 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:55.607179 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1261/2531] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:48:55 2022] POST /identity/v3/projects => generated 383 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:48:55.608825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39d47d-3821-4c10-bd30-8fc21615b299 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:55.631649 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39d47d-3821-4c10-bd30-8fc21615b299 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:55.789281 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80abff87-b096-4265-a2f0-6f004269a2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:55.790939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abff87-b096-4265-a2f0-6f004269a2bc admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:55.791704 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abff87-b096-4265-a2f0-6f004269a2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:55.792445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abff87-b096-4265-a2f0-6f004269a2bc admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:55.793761 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80abff87-b096-4265-a2f0-6f004269a2bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:55.797609 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80abff87-b096-4265-a2f0-6f004269a2bc admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:55.916440 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1271/2532] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:48:55 2022] POST /identity/v3/users => generated 298 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:48:55.920241 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cafcdc8-db02-4f8e-9c22-d2f30bb4bc99 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:55.943181 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cafcdc8-db02-4f8e-9c22-d2f30bb4bc99 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:55.970391 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d582c-f95f-4326-8902-994f5c598995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:55.972336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d582c-f95f-4326-8902-994f5c598995 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:55.972626 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d582c-f95f-4326-8902-994f5c598995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:55.974046 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d582c-f95f-4326-8902-994f5c598995 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:55.974709 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d582c-f95f-4326-8902-994f5c598995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:55.979354 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d582c-f95f-4326-8902-994f5c598995 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:56.000847 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1262/2533] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:56.005428 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80abff87-b096-4265-a2f0-6f004269a2bc admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:56.021790 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80abff87-b096-4265-a2f0-6f004269a2bc admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:56.027919 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873516ae-a64f-404a-b3ee-09c45167bb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:56.028628 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873516ae-a64f-404a-b3ee-09c45167bb8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:56.028960 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873516ae-a64f-404a-b3ee-09c45167bb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:56.029274 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873516ae-a64f-404a-b3ee-09c45167bb8f admin admin] PATH_INFO: `/v3/projects/4e510e9cdaa4457982ce1bc7af9aeee1/users/537bcc92d2b34c1c865708fa74c69bf3/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:56.052379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873516ae-a64f-404a-b3ee-09c45167bb8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e510e9cdaa4457982ce1bc7af9aeee1, user_id=537bcc92d2b34c1c865708fa74c69bf3, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:56.055866 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873516ae-a64f-404a-b3ee-09c45167bb8f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:56.166126 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1272/2534] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:48:56 2022] PUT /identity/v3/projects/4e510e9cdaa4457982ce1bc7af9aeee1/users/537bcc92d2b34c1c865708fa74c69bf3/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:56.170467 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d582c-f95f-4326-8902-994f5c598995 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:56.182747 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d582c-f95f-4326-8902-994f5c598995 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:56.330304 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c889dc8-fc4b-4c1a-8ea0-0ff5dfe0b96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:56.330304 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c889dc8-fc4b-4c1a-8ea0-0ff5dfe0b96f admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:56.330304 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c889dc8-fc4b-4c1a-8ea0-0ff5dfe0b96f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:56.330304 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c889dc8-fc4b-4c1a-8ea0-0ff5dfe0b96f admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:56.331187 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c889dc8-fc4b-4c1a-8ea0-0ff5dfe0b96f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:56.331187 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c889dc8-fc4b-4c1a-8ea0-0ff5dfe0b96f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:56.331187 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1263/2535] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:56 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:48:56.333379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873516ae-a64f-404a-b3ee-09c45167bb8f admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:56.344389 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873516ae-a64f-404a-b3ee-09c45167bb8f admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:56.348965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9eb668-9408-4174-8435-19dedbc24873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:56.349645 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9eb668-9408-4174-8435-19dedbc24873 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:56.349925 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9eb668-9408-4174-8435-19dedbc24873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:56.350209 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9eb668-9408-4174-8435-19dedbc24873 admin admin] PATH_INFO: `/v3/projects/4e510e9cdaa4457982ce1bc7af9aeee1/users/537bcc92d2b34c1c865708fa74c69bf3/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:56.352949 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9eb668-9408-4174-8435-19dedbc24873 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e510e9cdaa4457982ce1bc7af9aeee1, user_id=537bcc92d2b34c1c865708fa74c69bf3, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:56.355882 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9eb668-9408-4174-8435-19dedbc24873 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:56.358956 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:56.366248 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1273/2536] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:48:56 2022] PUT /identity/v3/projects/4e510e9cdaa4457982ce1bc7af9aeee1/users/537bcc92d2b34c1c865708fa74c69bf3/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:56.377840 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a988b8-16df-41e6-94d3-9126bad8ea05 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:56.378168 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a988b8-16df-41e6-94d3-9126bad8ea05 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:56.378476 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a988b8-16df-41e6-94d3-9126bad8ea05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:56.406506 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-66a988b8-16df-41e6-94d3-9126bad8ea05 None None] MFA Rules not processed for user `537bcc92d2b34c1c865708fa74c69bf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:56.407702 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-66a988b8-16df-41e6-94d3-9126bad8ea05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:56.421914 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:56.482102 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1264/2537] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:48:56.493130 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9eb668-9408-4174-8435-19dedbc24873 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:56.494968 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8a9eb668-9408-4174-8435-19dedbc24873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:56.562053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9eb668-9408-4174-8435-19dedbc24873 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:56.567099 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75f741-2618-443e-ab63-49d0653a19ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:56.567721 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75f741-2618-443e-ab63-49d0653a19ab service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:56.568065 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75f741-2618-443e-ab63-49d0653a19ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:56.568340 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75f741-2618-443e-ab63-49d0653a19ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:56.638162 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c75f741-2618-443e-ab63-49d0653a19ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:56.640395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c75f741-2618-443e-ab63-49d0653a19ab service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:56.654149 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1274/2538] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:48:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:48:56.864761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a988b8-16df-41e6-94d3-9126bad8ea05 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:56.867840 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-66a988b8-16df-41e6-94d3-9126bad8ea05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:56.954978 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a988b8-16df-41e6-94d3-9126bad8ea05 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:56.959934 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116e6da-057f-43f7-aa4c-5aca29aaa510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:56.961466 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116e6da-057f-43f7-aa4c-5aca29aaa510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:56.961854 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116e6da-057f-43f7-aa4c-5aca29aaa510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:56.962403 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116e6da-057f-43f7-aa4c-5aca29aaa510 admin admin] PATH_INFO: `/v3/users/98638c29485a455eb6fb99f396ec552d` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:56.963856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7116e6da-057f-43f7-aa4c-5aca29aaa510 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98638c29485a455eb6fb99f396ec552d)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:56.966080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7116e6da-057f-43f7-aa4c-5aca29aaa510 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:57.187398 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-7116e6da-057f-43f7-aa4c-5aca29aaa510 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98638c29485a455eb6fb99f396ec552d'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:57.188288 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-7116e6da-057f-43f7-aa4c-5aca29aaa510 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98638c29485a455eb6fb99f396ec552d'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:57.190119 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-7116e6da-057f-43f7-aa4c-5aca29aaa510 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98638c29485a455eb6fb99f396ec552d'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:57.216540 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-7116e6da-057f-43f7-aa4c-5aca29aaa510 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98638c29485a455eb6fb99f396ec552d'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:57.271825 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-7116e6da-057f-43f7-aa4c-5aca29aaa510 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98638c29485a455eb6fb99f396ec552d'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:57.296708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1265/2539] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:48:56 2022] DELETE /identity/v3/users/98638c29485a455eb6fb99f396ec552d => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:57.301266 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4b742-b80c-43fc-8553-f6bee4e753e8 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:57.301561 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4b742-b80c-43fc-8553-f6bee4e753e8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:57.301844 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4b742-b80c-43fc-8553-f6bee4e753e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:57.330214 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-24d4b742-b80c-43fc-8553-f6bee4e753e8 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:57.336730 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-24d4b742-b80c-43fc-8553-f6bee4e753e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:57.475356 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1275/2540] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:48:57.483338 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116e6da-057f-43f7-aa4c-5aca29aaa510 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:57.485133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-7116e6da-057f-43f7-aa4c-5aca29aaa510 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:57.535256 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:57.588802 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116e6da-057f-43f7-aa4c-5aca29aaa510 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:57.593004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a404419-b086-4ae2-b644-8e5d77c4795d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:57.593820 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a404419-b086-4ae2-b644-8e5d77c4795d service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:57.594229 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a404419-b086-4ae2-b644-8e5d77c4795d service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:57.594586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a404419-b086-4ae2-b644-8e5d77c4795d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:57.606395 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a404419-b086-4ae2-b644-8e5d77c4795d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:57.607943 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a404419-b086-4ae2-b644-8e5d77c4795d service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:57.621649 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1266/2541] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:48:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:48:58.161679 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4b742-b80c-43fc-8553-f6bee4e753e8 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:58.164501 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-24d4b742-b80c-43fc-8553-f6bee4e753e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:58.183743 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4b742-b80c-43fc-8553-f6bee4e753e8 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:58.193881 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c6d7f-f087-46e1-8811-20d8fe3849bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:58.195424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c6d7f-f087-46e1-8811-20d8fe3849bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:58.196017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c6d7f-f087-46e1-8811-20d8fe3849bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:58.196532 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c6d7f-f087-46e1-8811-20d8fe3849bf admin admin] PATH_INFO: `/v3/projects/752103b10916470eb60d43a9d0fb45a3` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:58.197966 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13c6d7f-f087-46e1-8811-20d8fe3849bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=752103b10916470eb60d43a9d0fb45a3)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:58.208110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13c6d7f-f087-46e1-8811-20d8fe3849bf admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:58.333535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:58.339819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b13c6d7f-f087-46e1-8811-20d8fe3849bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '752103b10916470eb60d43a9d0fb45a3', 'request_id': 'req-b13c6d7f-f087-46e1-8811-20d8fe3849bf'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:58.414746 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b13c6d7f-f087-46e1-8811-20d8fe3849bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '752103b10916470eb60d43a9d0fb45a3', 'request_id': 'req-b13c6d7f-f087-46e1-8811-20d8fe3849bf'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:58.433402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b13c6d7f-f087-46e1-8811-20d8fe3849bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '752103b10916470eb60d43a9d0fb45a3', 'request_id': 'req-b13c6d7f-f087-46e1-8811-20d8fe3849bf'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:58.452260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b13c6d7f-f087-46e1-8811-20d8fe3849bf admin admin] The token cache is being invalidate because project 752103b10916470eb60d43a9d0fb45a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:48:58.452772 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b13c6d7f-f087-46e1-8811-20d8fe3849bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:58.453755 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-b13c6d7f-f087-46e1-8811-20d8fe3849bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:48:58.460135 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1276/2542] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:48:58 2022] DELETE /identity/v3/projects/752103b10916470eb60d43a9d0fb45a3 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:58.462583 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36bc80-4eef-4674-9ad7-d71574632263 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:58.462882 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36bc80-4eef-4674-9ad7-d71574632263 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:58.463168 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36bc80-4eef-4674-9ad7-d71574632263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:58.485368 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-cf36bc80-4eef-4674-9ad7-d71574632263 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:58.486413 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-cf36bc80-4eef-4674-9ad7-d71574632263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:58.564117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1267/2543] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:58.572723 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a826dc9-753a-4047-8c95-5c672c0512e2 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:58.573012 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a826dc9-753a-4047-8c95-5c672c0512e2 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:58.573328 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a826dc9-753a-4047-8c95-5c672c0512e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:58.595625 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8a826dc9-753a-4047-8c95-5c672c0512e2 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:58.596711 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8a826dc9-753a-4047-8c95-5c672c0512e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:58.610839 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36bc80-4eef-4674-9ad7-d71574632263 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:58.615758 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1277/2544] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:58.622832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36bc80-4eef-4674-9ad7-d71574632263 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:58.629275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c405c5-d6a0-4c37-aa3f-db6ec0f66b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:58.630041 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c405c5-d6a0-4c37-aa3f-db6ec0f66b12 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:58.630769 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c405c5-d6a0-4c37-aa3f-db6ec0f66b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:58.631009 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c405c5-d6a0-4c37-aa3f-db6ec0f66b12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:58.631698 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c405c5-d6a0-4c37-aa3f-db6ec0f66b12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:48:58.632666 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c405c5-d6a0-4c37-aa3f-db6ec0f66b12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:58.634314 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c405c5-d6a0-4c37-aa3f-db6ec0f66b12 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:58.649819 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1268/2545] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:48:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:48:58.654060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a826dc9-753a-4047-8c95-5c672c0512e2 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:58.664247 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a826dc9-753a-4047-8c95-5c672c0512e2 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:58.669697 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd9464-6911-4dd3-a17a-cc2cfdb8db3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:58.670315 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd9464-6911-4dd3-a17a-cc2cfdb8db3e admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:58.670594 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd9464-6911-4dd3-a17a-cc2cfdb8db3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:58.670865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd9464-6911-4dd3-a17a-cc2cfdb8db3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:58.671502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd9464-6911-4dd3-a17a-cc2cfdb8db3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:58.673442 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd9464-6911-4dd3-a17a-cc2cfdb8db3e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:58.785448 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1278/2546] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:48:58 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:48:58.787345 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c405c5-d6a0-4c37-aa3f-db6ec0f66b12 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:58.799842 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c405c5-d6a0-4c37-aa3f-db6ec0f66b12 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:58.876876 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29665db3-c1d5-4f32-96b6-d97fd3f287bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:58.877540 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29665db3-c1d5-4f32-96b6-d97fd3f287bb admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:58.877820 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29665db3-c1d5-4f32-96b6-d97fd3f287bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:58.878099 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29665db3-c1d5-4f32-96b6-d97fd3f287bb admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:58.879191 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29665db3-c1d5-4f32-96b6-d97fd3f287bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:58.881414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29665db3-c1d5-4f32-96b6-d97fd3f287bb admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:58.992602 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1269/2547] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:48:58 2022] POST /identity/v3/users => generated 292 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:48:58.997271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd9464-6911-4dd3-a17a-cc2cfdb8db3e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:59.015708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd9464-6911-4dd3-a17a-cc2cfdb8db3e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:59.028819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30f9a5-c8be-49cf-b739-7495424f4feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:59.030258 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30f9a5-c8be-49cf-b739-7495424f4feb admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:59.030828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30f9a5-c8be-49cf-b739-7495424f4feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:59.031151 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30f9a5-c8be-49cf-b739-7495424f4feb admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:59.032398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c30f9a5-c8be-49cf-b739-7495424f4feb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:59.034830 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c30f9a5-c8be-49cf-b739-7495424f4feb admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:59.050901 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1279/2548] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:58 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:59.055747 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29665db3-c1d5-4f32-96b6-d97fd3f287bb admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:59.067230 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29665db3-c1d5-4f32-96b6-d97fd3f287bb admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:59.072981 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad18a6-1fb4-4014-8c9e-0d578981b1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:59.074760 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad18a6-1fb4-4014-8c9e-0d578981b1ef admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:59.074760 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad18a6-1fb4-4014-8c9e-0d578981b1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:59.074760 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad18a6-1fb4-4014-8c9e-0d578981b1ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:59.075109 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad18a6-1fb4-4014-8c9e-0d578981b1ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:59.076490 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad18a6-1fb4-4014-8c9e-0d578981b1ef admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:59.093609 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1270/2549] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:48:59.098388 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30f9a5-c8be-49cf-b739-7495424f4feb admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:59.110891 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30f9a5-c8be-49cf-b739-7495424f4feb admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:59.115795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d705028-f9a9-4f45-beed-bd7e83bdb16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:59.117426 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d705028-f9a9-4f45-beed-bd7e83bdb16d admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:59.117426 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d705028-f9a9-4f45-beed-bd7e83bdb16d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:59.117971 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d705028-f9a9-4f45-beed-bd7e83bdb16d admin admin] PATH_INFO: `/v3/projects/e016f91b0fb44e15881edccaaf67decf/users/52816d7a61f74a9a96b55d0e78147c7b/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:59.141172 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d705028-f9a9-4f45-beed-bd7e83bdb16d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e016f91b0fb44e15881edccaaf67decf, user_id=52816d7a61f74a9a96b55d0e78147c7b, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:59.144753 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d705028-f9a9-4f45-beed-bd7e83bdb16d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:59.260291 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1280/2550] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:48:59 2022] PUT /identity/v3/projects/e016f91b0fb44e15881edccaaf67decf/users/52816d7a61f74a9a96b55d0e78147c7b/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:48:59.274832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab78091-fc7f-454a-96c3-f4b4aad9a3e0 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:59.275450 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab78091-fc7f-454a-96c3-f4b4aad9a3e0 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:59.275893 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab78091-fc7f-454a-96c3-f4b4aad9a3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:59.318051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-5ab78091-fc7f-454a-96c3-f4b4aad9a3e0 None None] MFA Rules not processed for user `52816d7a61f74a9a96b55d0e78147c7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:48:59.319344 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5ab78091-fc7f-454a-96c3-f4b4aad9a3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:48:59.352183 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:48:59.400357 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1271/2551] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:48:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:48:59.404906 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d705028-f9a9-4f45-beed-bd7e83bdb16d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:59.485874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d705028-f9a9-4f45-beed-bd7e83bdb16d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:59.490645 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62306d-d08e-4fcd-a9a1-21b10f6092c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:59.491320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62306d-d08e-4fcd-a9a1-21b10f6092c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:59.491675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62306d-d08e-4fcd-a9a1-21b10f6092c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:59.491997 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62306d-d08e-4fcd-a9a1-21b10f6092c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:59.492656 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62306d-d08e-4fcd-a9a1-21b10f6092c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:59.494867 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62306d-d08e-4fcd-a9a1-21b10f6092c8 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:59.728376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1281/2552] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:48:59 2022] POST /identity/v3/projects => generated 383 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:48:59.729519 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab78091-fc7f-454a-96c3-f4b4aad9a3e0 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:59.741755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab78091-fc7f-454a-96c3-f4b4aad9a3e0 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:59.795871 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a35870-7d3d-4ad8-b079-4a76d62dada0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:59.796530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a35870-7d3d-4ad8-b079-4a76d62dada0 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:59.796812 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a35870-7d3d-4ad8-b079-4a76d62dada0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:59.797082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a35870-7d3d-4ad8-b079-4a76d62dada0 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:59.797728 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a35870-7d3d-4ad8-b079-4a76d62dada0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:59.799741 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a35870-7d3d-4ad8-b079-4a76d62dada0 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:59.918075 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1272/2553] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:48:59 2022] POST /identity/v3/users => generated 298 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:48:59.922052 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62306d-d08e-4fcd-a9a1-21b10f6092c8 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:59.933785 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62306d-d08e-4fcd-a9a1-21b10f6092c8 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:59.939036 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74428cef-3a2e-427f-bd09-59990a04cd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:59.939746 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74428cef-3a2e-427f-bd09-59990a04cd93 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:59.940043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74428cef-3a2e-427f-bd09-59990a04cd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:59.940489 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74428cef-3a2e-427f-bd09-59990a04cd93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:59.941170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74428cef-3a2e-427f-bd09-59990a04cd93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:48:59.943149 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74428cef-3a2e-427f-bd09-59990a04cd93 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:48:59.956142 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1282/2554] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:48:59.959961 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a35870-7d3d-4ad8-b079-4a76d62dada0 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:48:59.972692 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a35870-7d3d-4ad8-b079-4a76d62dada0 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:48:59.977769 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf1c440-c416-4481-b283-82816092c7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:48:59.978739 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1c440-c416-4481-b283-82816092c7e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:48:59.979027 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1c440-c416-4481-b283-82816092c7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:48:59.979333 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1c440-c416-4481-b283-82816092c7e9 admin admin] PATH_INFO: `/v3/projects/f71680e7cc3943cfa16b61d8d4d87319/users/82878b76d3fb447b8cce08b5e075f18c/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:48:59.995021 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf1c440-c416-4481-b283-82816092c7e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f71680e7cc3943cfa16b61d8d4d87319, user_id=82878b76d3fb447b8cce08b5e075f18c, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:00.001662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf1c440-c416-4481-b283-82816092c7e9 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:00.082431 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1273/2555] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:48:59 2022] PUT /identity/v3/projects/f71680e7cc3943cfa16b61d8d4d87319/users/82878b76d3fb447b8cce08b5e075f18c/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:00.086478 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74428cef-3a2e-427f-bd09-59990a04cd93 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:00.096708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74428cef-3a2e-427f-bd09-59990a04cd93 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:00.138905 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:00.153770 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f2d61-81a9-47e7-84e5-29cd8158cc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:00.154401 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f2d61-81a9-47e7-84e5-29cd8158cc2c admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:00.154681 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f2d61-81a9-47e7-84e5-29cd8158cc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:00.155014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f2d61-81a9-47e7-84e5-29cd8158cc2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:00.155655 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f2d61-81a9-47e7-84e5-29cd8158cc2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:00.157282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f2d61-81a9-47e7-84e5-29cd8158cc2c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:00.167607 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1283/2556] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:49:00.171860 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf1c440-c416-4481-b283-82816092c7e9 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:00.182890 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf1c440-c416-4481-b283-82816092c7e9 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:00.187547 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58dec11-1380-416c-b612-515e20c597c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:00.188192 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58dec11-1380-416c-b612-515e20c597c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:00.188472 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58dec11-1380-416c-b612-515e20c597c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:00.188745 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58dec11-1380-416c-b612-515e20c597c5 admin admin] PATH_INFO: `/v3/projects/f71680e7cc3943cfa16b61d8d4d87319/users/82878b76d3fb447b8cce08b5e075f18c/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:00.190757 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58dec11-1380-416c-b612-515e20c597c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f71680e7cc3943cfa16b61d8d4d87319, user_id=82878b76d3fb447b8cce08b5e075f18c, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:00.193403 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58dec11-1380-416c-b612-515e20c597c5 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:00.204246 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1274/2557] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:49:00 2022] PUT /identity/v3/projects/f71680e7cc3943cfa16b61d8d4d87319/users/82878b76d3fb447b8cce08b5e075f18c/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:00.214954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8514d66-fd3e-4bbb-90e9-ddf780c62f5e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:00.215243 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8514d66-fd3e-4bbb-90e9-ddf780c62f5e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:00.215515 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8514d66-fd3e-4bbb-90e9-ddf780c62f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:00.243524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e8514d66-fd3e-4bbb-90e9-ddf780c62f5e None None] MFA Rules not processed for user `82878b76d3fb447b8cce08b5e075f18c`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:00.244683 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e8514d66-fd3e-4bbb-90e9-ddf780c62f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:00.311935 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1284/2558] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:00 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:49:00.322565 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58dec11-1380-416c-b612-515e20c597c5 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:00.324278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d58dec11-1380-416c-b612-515e20c597c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:00.381735 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58dec11-1380-416c-b612-515e20c597c5 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:00.385536 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d268190-c90d-4a5e-896d-8719e9f65f63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:00.386128 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d268190-c90d-4a5e-896d-8719e9f65f63 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:00.386404 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d268190-c90d-4a5e-896d-8719e9f65f63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:00.386672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d268190-c90d-4a5e-896d-8719e9f65f63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:00.395335 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d268190-c90d-4a5e-896d-8719e9f65f63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:00.396742 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d268190-c90d-4a5e-896d-8719e9f65f63 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:00.406301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1275/2559] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:49:00.510238 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8514d66-fd3e-4bbb-90e9-ddf780c62f5e None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:00.511742 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e8514d66-fd3e-4bbb-90e9-ddf780c62f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:00.525079 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8514d66-fd3e-4bbb-90e9-ddf780c62f5e None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:00.528762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55a9e6-47d7-42f0-92fb-da9ba93de020 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:00.529427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55a9e6-47d7-42f0-92fb-da9ba93de020 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:00.529746 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55a9e6-47d7-42f0-92fb-da9ba93de020 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:00.530061 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55a9e6-47d7-42f0-92fb-da9ba93de020 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:00.596982 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a55a9e6-47d7-42f0-92fb-da9ba93de020 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:00.599019 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a55a9e6-47d7-42f0-92fb-da9ba93de020 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:00.611121 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1285/2560] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:49:03.222259 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d268190-c90d-4a5e-896d-8719e9f65f63 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:03.232761 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d268190-c90d-4a5e-896d-8719e9f65f63 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:03.237361 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51327b-0c62-4dc3-b662-c29dd4fba211 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:03.242029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51327b-0c62-4dc3-b662-c29dd4fba211 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:03.242029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51327b-0c62-4dc3-b662-c29dd4fba211 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:03.242029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51327b-0c62-4dc3-b662-c29dd4fba211 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:03.242029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-3a51327b-0c62-4dc3-b662-c29dd4fba211 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:03.265213 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:03.317886 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51327b-0c62-4dc3-b662-c29dd4fba211 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:03.320040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51327b-0c62-4dc3-b662-c29dd4fba211 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:03.337869 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1276/2561] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:03 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:49:03.808669 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55a9e6-47d7-42f0-92fb-da9ba93de020 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:03.880712 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55a9e6-47d7-42f0-92fb-da9ba93de020 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:03.888958 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23734ca9-d013-463b-b552-7ed9e2605c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:03.890034 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23734ca9-d013-463b-b552-7ed9e2605c47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:03.890323 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23734ca9-d013-463b-b552-7ed9e2605c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:03.890665 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23734ca9-d013-463b-b552-7ed9e2605c47 admin admin] PATH_INFO: `/v3/users/52816d7a61f74a9a96b55d0e78147c7b` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:03.892212 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23734ca9-d013-463b-b552-7ed9e2605c47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52816d7a61f74a9a96b55d0e78147c7b)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:03.894479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23734ca9-d013-463b-b552-7ed9e2605c47 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:03.902307 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:04.069728 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-23734ca9-d013-463b-b552-7ed9e2605c47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52816d7a61f74a9a96b55d0e78147c7b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:04.071281 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-23734ca9-d013-463b-b552-7ed9e2605c47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52816d7a61f74a9a96b55d0e78147c7b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:04.072181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-23734ca9-d013-463b-b552-7ed9e2605c47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52816d7a61f74a9a96b55d0e78147c7b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:04.082872 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-23734ca9-d013-463b-b552-7ed9e2605c47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52816d7a61f74a9a96b55d0e78147c7b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:04.156939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-23734ca9-d013-463b-b552-7ed9e2605c47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52816d7a61f74a9a96b55d0e78147c7b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:04.206703 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1286/2562] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:49:03 2022] DELETE /identity/v3/users/52816d7a61f74a9a96b55d0e78147c7b => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:04.216735 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2802189-e40d-440a-b562-d62786a01df1 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:04.216850 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2802189-e40d-440a-b562-d62786a01df1 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:04.217126 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2802189-e40d-440a-b562-d62786a01df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:04.240236 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-b2802189-e40d-440a-b562-d62786a01df1 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:04.241453 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b2802189-e40d-440a-b562-d62786a01df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:04.322940 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1277/2563] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:04.333977 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23734ca9-d013-463b-b552-7ed9e2605c47 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:04.336033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-23734ca9-d013-463b-b552-7ed9e2605c47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:04.413653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23734ca9-d013-463b-b552-7ed9e2605c47 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:04.429942 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9fa7e-4033-401a-a05e-cf1f41cc3fde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:04.430594 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9fa7e-4033-401a-a05e-cf1f41cc3fde service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:04.430877 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9fa7e-4033-401a-a05e-cf1f41cc3fde service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:04.431162 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9fa7e-4033-401a-a05e-cf1f41cc3fde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:04.443940 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd9fa7e-4033-401a-a05e-cf1f41cc3fde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:04.446561 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd9fa7e-4033-401a-a05e-cf1f41cc3fde service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:04.457389 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1287/2564] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:49:04.963856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2802189-e40d-440a-b562-d62786a01df1 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:04.965349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b2802189-e40d-440a-b562-d62786a01df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:04.978764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2802189-e40d-440a-b562-d62786a01df1 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:04.983418 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25941553-894b-458c-81b2-e0b6d179d5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:04.984062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25941553-894b-458c-81b2-e0b6d179d5bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:04.984360 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25941553-894b-458c-81b2-e0b6d179d5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:04.984616 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25941553-894b-458c-81b2-e0b6d179d5bf admin admin] PATH_INFO: `/v3/projects/e016f91b0fb44e15881edccaaf67decf` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:04.985752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25941553-894b-458c-81b2-e0b6d179d5bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e016f91b0fb44e15881edccaaf67decf)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:04.987656 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25941553-894b-458c-81b2-e0b6d179d5bf admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:05.130284 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-25941553-894b-458c-81b2-e0b6d179d5bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e016f91b0fb44e15881edccaaf67decf', 'request_id': 'req-25941553-894b-458c-81b2-e0b6d179d5bf'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:05.137044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-25941553-894b-458c-81b2-e0b6d179d5bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e016f91b0fb44e15881edccaaf67decf', 'request_id': 'req-25941553-894b-458c-81b2-e0b6d179d5bf'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:05.142834 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-25941553-894b-458c-81b2-e0b6d179d5bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e016f91b0fb44e15881edccaaf67decf', 'request_id': 'req-25941553-894b-458c-81b2-e0b6d179d5bf'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:05.221069 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-25941553-894b-458c-81b2-e0b6d179d5bf admin admin] The token cache is being invalidate because project e016f91b0fb44e15881edccaaf67decf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:49:05.221482 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-25941553-894b-458c-81b2-e0b6d179d5bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:05.222358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-25941553-894b-458c-81b2-e0b6d179d5bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:05.224865 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1278/2565] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:49:04 2022] DELETE /identity/v3/projects/e016f91b0fb44e15881edccaaf67decf => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:05.229239 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9fa7e-4033-401a-a05e-cf1f41cc3fde service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:05.247805 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9fa7e-4033-401a-a05e-cf1f41cc3fde service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:05.317083 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26178dad-fe24-47ab-ad4b-1d9c9bb8340c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:05.317769 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26178dad-fe24-47ab-ad4b-1d9c9bb8340c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:05.318051 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26178dad-fe24-47ab-ad4b-1d9c9bb8340c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:05.318325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26178dad-fe24-47ab-ad4b-1d9c9bb8340c admin admin] PATH_INFO: `/v3/users/82878b76d3fb447b8cce08b5e075f18c` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:05.320129 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26178dad-fe24-47ab-ad4b-1d9c9bb8340c admin admin] RBAC: Authorizing `identity:delete_user(user_id=82878b76d3fb447b8cce08b5e075f18c)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:05.322409 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26178dad-fe24-47ab-ad4b-1d9c9bb8340c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:05.410118 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:05.481351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-26178dad-fe24-47ab-ad4b-1d9c9bb8340c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82878b76d3fb447b8cce08b5e075f18c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:05.482012 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-26178dad-fe24-47ab-ad4b-1d9c9bb8340c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82878b76d3fb447b8cce08b5e075f18c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:05.482615 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-26178dad-fe24-47ab-ad4b-1d9c9bb8340c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82878b76d3fb447b8cce08b5e075f18c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:05.492474 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-26178dad-fe24-47ab-ad4b-1d9c9bb8340c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82878b76d3fb447b8cce08b5e075f18c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:05.567090 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-26178dad-fe24-47ab-ad4b-1d9c9bb8340c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82878b76d3fb447b8cce08b5e075f18c'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:05.585461 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1288/2566] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:49:05 2022] DELETE /identity/v3/users/82878b76d3fb447b8cce08b5e075f18c => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:49:06.060752 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25941553-894b-458c-81b2-e0b6d179d5bf admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:06.062223 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-25941553-894b-458c-81b2-e0b6d179d5bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:06.075388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25941553-894b-458c-81b2-e0b6d179d5bf admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:06.113365 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:06.136133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c148f-088f-437a-a813-837ba3e65dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:06.136777 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c148f-088f-437a-a813-837ba3e65dc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:06.137059 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c148f-088f-437a-a813-837ba3e65dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:06.137364 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c148f-088f-437a-a813-837ba3e65dc1 admin admin] PATH_INFO: `/v3/projects/f71680e7cc3943cfa16b61d8d4d87319` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:06.138810 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c148f-088f-437a-a813-837ba3e65dc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f71680e7cc3943cfa16b61d8d4d87319)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:06.141454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c148f-088f-437a-a813-837ba3e65dc1 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:06.249388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d26c148f-088f-437a-a813-837ba3e65dc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f71680e7cc3943cfa16b61d8d4d87319', 'request_id': 'req-d26c148f-088f-437a-a813-837ba3e65dc1'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:06.258233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d26c148f-088f-437a-a813-837ba3e65dc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f71680e7cc3943cfa16b61d8d4d87319', 'request_id': 'req-d26c148f-088f-437a-a813-837ba3e65dc1'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:06.267193 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d26c148f-088f-437a-a813-837ba3e65dc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f71680e7cc3943cfa16b61d8d4d87319', 'request_id': 'req-d26c148f-088f-437a-a813-837ba3e65dc1'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:06.329743 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d26c148f-088f-437a-a813-837ba3e65dc1 admin admin] The token cache is being invalidate because project f71680e7cc3943cfa16b61d8d4d87319 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:49:06.329743 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d26c148f-088f-437a-a813-837ba3e65dc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:06.329743 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d26c148f-088f-437a-a813-837ba3e65dc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:06.330962 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1279/2567] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:49:06 2022] DELETE /identity/v3/projects/f71680e7cc3943cfa16b61d8d4d87319 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:06.336453 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8ece8-0303-4029-8936-fb881f77d486 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:06.336970 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8ece8-0303-4029-8936-fb881f77d486 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:06.337060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8ece8-0303-4029-8936-fb881f77d486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:06.358269 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0cd8ece8-0303-4029-8936-fb881f77d486 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:06.359498 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0cd8ece8-0303-4029-8936-fb881f77d486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:06.368934 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86e656-3f03-437a-9d08-dd187fd7436f None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:06.369247 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86e656-3f03-437a-9d08-dd187fd7436f None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:06.369574 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86e656-3f03-437a-9d08-dd187fd7436f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:06.388990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-6e86e656-3f03-437a-9d08-dd187fd7436f None None] MFA Rules not processed for user `7f3422c6f9a041ceba851b71bd15d0df`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:06.390215 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6e86e656-3f03-437a-9d08-dd187fd7436f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:06.429595 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1289/2568] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:06.437319 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e240b-0666-44e8-ac80-a61e814564e8 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:06.437605 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e240b-0666-44e8-ac80-a61e814564e8 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:06.437879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e240b-0666-44e8-ac80-a61e814564e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:06.452472 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1280/2569] 10.222.0.33 () {58 vars in 967 bytes} [Mon Aug 8 08:49:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:49:06.458342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c95e240b-0666-44e8-ac80-a61e814564e8 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:06.458342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c95e240b-0666-44e8-ac80-a61e814564e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:06.465314 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e86e656-3f03-437a-9d08-dd187fd7436f None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:06.466399 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1290/2570] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:49:06.469894 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6e86e656-3f03-437a-9d08-dd187fd7436f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:06.470451 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e240b-0666-44e8-ac80-a61e814564e8 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:06.480573 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e240b-0666-44e8-ac80-a61e814564e8 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:06.486212 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c77a947-5953-47b8-a8a7-500dc46609a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:06.486835 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77a947-5953-47b8-a8a7-500dc46609a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:06.487523 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77a947-5953-47b8-a8a7-500dc46609a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:06.487800 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77a947-5953-47b8-a8a7-500dc46609a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:06.488446 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c77a947-5953-47b8-a8a7-500dc46609a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:49:06.488751 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c77a947-5953-47b8-a8a7-500dc46609a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:06.490368 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c77a947-5953-47b8-a8a7-500dc46609a7 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:06.502656 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1291/2571] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:49:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:49:06.507311 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c77a947-5953-47b8-a8a7-500dc46609a7 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:06.517671 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c77a947-5953-47b8-a8a7-500dc46609a7 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:06.522632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d1c02-3565-436a-805a-f185ac7fb035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:06.523567 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d1c02-3565-436a-805a-f185ac7fb035 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:06.523567 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d1c02-3565-436a-805a-f185ac7fb035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:06.525439 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d1c02-3565-436a-805a-f185ac7fb035 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:06.525439 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d1c02-3565-436a-805a-f185ac7fb035 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:06.526330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d1c02-3565-436a-805a-f185ac7fb035 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:06.541483 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e86e656-3f03-437a-9d08-dd187fd7436f None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:06.546476 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0121b382-f485-49d5-83d8-3001682814bf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e69c0e02ad944f2a85fc5318018e5cb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:06.547085 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121b382-f485-49d5-83d8-3001682814bf service swift] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:06.547361 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121b382-f485-49d5-83d8-3001682814bf service swift] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:06.547629 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121b382-f485-49d5-83d8-3001682814bf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:06.555923 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0121b382-f485-49d5-83d8-3001682814bf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:06.557542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0121b382-f485-49d5-83d8-3001682814bf service swift] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:06.565532 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1281/2572] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 08:49:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 08:49:06.638505 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1292/2573] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:49:06 2022] POST /identity/v3/projects => generated 375 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:49:06.642757 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0121b382-f485-49d5-83d8-3001682814bf service swift] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:06.648705 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:06.654723 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0121b382-f485-49d5-83d8-3001682814bf service swift] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:06.712704 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab18912a-f929-4d85-8196-51d12ab8e9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:06.713350 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18912a-f929-4d85-8196-51d12ab8e9ad admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:06.713672 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18912a-f929-4d85-8196-51d12ab8e9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:06.713897 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18912a-f929-4d85-8196-51d12ab8e9ad admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:06.714529 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab18912a-f929-4d85-8196-51d12ab8e9ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:06.716428 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab18912a-f929-4d85-8196-51d12ab8e9ad admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:06.820250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1282/2574] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:49:06 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:49:06.824449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d1c02-3565-436a-805a-f185ac7fb035 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:06.835182 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d1c02-3565-436a-805a-f185ac7fb035 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:06.839862 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77141d3e-b0f8-43f2-8390-bbb7beb47523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:06.840458 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77141d3e-b0f8-43f2-8390-bbb7beb47523 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:06.840735 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77141d3e-b0f8-43f2-8390-bbb7beb47523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:06.841007 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77141d3e-b0f8-43f2-8390-bbb7beb47523 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:06.841682 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77141d3e-b0f8-43f2-8390-bbb7beb47523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:06.843281 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77141d3e-b0f8-43f2-8390-bbb7beb47523 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:06.853748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1293/2575] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:49:06.861091 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab18912a-f929-4d85-8196-51d12ab8e9ad admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:06.869912 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab18912a-f929-4d85-8196-51d12ab8e9ad admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:06.874679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e4fc6-78bb-4374-9a0d-e45f3c0d5399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:06.875417 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e4fc6-78bb-4374-9a0d-e45f3c0d5399 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:06.875776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e4fc6-78bb-4374-9a0d-e45f3c0d5399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:06.876053 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e4fc6-78bb-4374-9a0d-e45f3c0d5399 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:06.876727 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518e4fc6-78bb-4374-9a0d-e45f3c0d5399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:06.881625 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518e4fc6-78bb-4374-9a0d-e45f3c0d5399 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:06.882774 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77141d3e-b0f8-43f2-8390-bbb7beb47523 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:06.889749 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-77141d3e-b0f8-43f2-8390-bbb7beb47523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:06.893701 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1283/2576] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:49:06.897985 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e4fc6-78bb-4374-9a0d-e45f3c0d5399 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:06.901549 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77141d3e-b0f8-43f2-8390-bbb7beb47523 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:06.908567 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e4fc6-78bb-4374-9a0d-e45f3c0d5399 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:06.910623 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05636f40-2049-4b86-a33f-b2d6a61b463a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:06.911466 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05636f40-2049-4b86-a33f-b2d6a61b463a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:06.911753 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05636f40-2049-4b86-a33f-b2d6a61b463a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:06.912026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05636f40-2049-4b86-a33f-b2d6a61b463a admin admin] PATH_INFO: `/v3/users/2102c960a4804973bfad9f17ec68d6d5` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:06.914036 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0842ca-cbbf-448f-a4a9-afce1b728c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:06.914790 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05636f40-2049-4b86-a33f-b2d6a61b463a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2102c960a4804973bfad9f17ec68d6d5)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:06.915006 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0842ca-cbbf-448f-a4a9-afce1b728c14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:06.915503 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0842ca-cbbf-448f-a4a9-afce1b728c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:06.915962 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0842ca-cbbf-448f-a4a9-afce1b728c14 admin admin] PATH_INFO: `/v3/projects/e32e3d331f154cd08d5e6d1e6c9225ee/users/b3ba063a311c458a8ef11424bea1c673/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:06.921200 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05636f40-2049-4b86-a33f-b2d6a61b463a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:06.936153 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0842ca-cbbf-448f-a4a9-afce1b728c14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e32e3d331f154cd08d5e6d1e6c9225ee, user_id=b3ba063a311c458a8ef11424bea1c673, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:06.938821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0842ca-cbbf-448f-a4a9-afce1b728c14 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:07.047850 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1284/2577] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:49:06 2022] PUT /identity/v3/projects/e32e3d331f154cd08d5e6d1e6c9225ee/users/b3ba063a311c458a8ef11424bea1c673/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:49:07.060292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d0932-b90d-4609-9d43-82a403753b05 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:07.060571 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d0932-b90d-4609-9d43-82a403753b05 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:07.060839 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d0932-b90d-4609-9d43-82a403753b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:07.092213 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-8a3d0932-b90d-4609-9d43-82a403753b05 None None] MFA Rules not processed for user `b3ba063a311c458a8ef11424bea1c673`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:07.093539 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8a3d0932-b90d-4609-9d43-82a403753b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:07.143952 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-05636f40-2049-4b86-a33f-b2d6a61b463a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2102c960a4804973bfad9f17ec68d6d5'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:07.144674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-05636f40-2049-4b86-a33f-b2d6a61b463a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2102c960a4804973bfad9f17ec68d6d5'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:07.145544 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-05636f40-2049-4b86-a33f-b2d6a61b463a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2102c960a4804973bfad9f17ec68d6d5'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:07.146897 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:07.153727 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-05636f40-2049-4b86-a33f-b2d6a61b463a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2102c960a4804973bfad9f17ec68d6d5'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:07.166432 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1285/2578] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:49:07.170817 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d0932-b90d-4609-9d43-82a403753b05 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:07.172127 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8a3d0932-b90d-4609-9d43-82a403753b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:07.222282 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:07.239245 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d0932-b90d-4609-9d43-82a403753b05 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:07.244166 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e70319-8af9-4ace-b47f-bf2609b18ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:07.244787 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e70319-8af9-4ace-b47f-bf2609b18ddb admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:07.245083 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e70319-8af9-4ace-b47f-bf2609b18ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:07.245373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e70319-8af9-4ace-b47f-bf2609b18ddb admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:07.246242 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e70319-8af9-4ace-b47f-bf2609b18ddb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:07.248188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e70319-8af9-4ace-b47f-bf2609b18ddb admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:07.248487 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-05636f40-2049-4b86-a33f-b2d6a61b463a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2102c960a4804973bfad9f17ec68d6d5'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:07.266872 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1294/2579] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:49:06 2022] DELETE /identity/v3/users/2102c960a4804973bfad9f17ec68d6d5 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:07.271255 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a75a1-099a-4e1e-aaa7-bd676e053cbd None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:07.271540 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a75a1-099a-4e1e-aaa7-bd676e053cbd None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:07.271814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a75a1-099a-4e1e-aaa7-bd676e053cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:07.292642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-8d9a75a1-099a-4e1e-aaa7-bd676e053cbd None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:07.293892 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8d9a75a1-099a-4e1e-aaa7-bd676e053cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:07.345407 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1286/2580] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:49:07 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:49:07.346267 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e70319-8af9-4ace-b47f-bf2609b18ddb admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:07.355169 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1295/2581] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:49:07.368595 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a75a1-099a-4e1e-aaa7-bd676e053cbd None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:07.369981 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8d9a75a1-099a-4e1e-aaa7-bd676e053cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:07.416210 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e70319-8af9-4ace-b47f-bf2609b18ddb admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:07.421565 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f033016-69ec-4c2d-b0cc-d71d9c16b94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:07.422064 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f033016-69ec-4c2d-b0cc-d71d9c16b94c admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:07.422344 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f033016-69ec-4c2d-b0cc-d71d9c16b94c admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:07.422618 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f033016-69ec-4c2d-b0cc-d71d9c16b94c admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:07.423260 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f033016-69ec-4c2d-b0cc-d71d9c16b94c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:07.425239 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f033016-69ec-4c2d-b0cc-d71d9c16b94c admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:07.433325 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a75a1-099a-4e1e-aaa7-bd676e053cbd None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:07.481434 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aecb05-3746-44b2-a88f-c91d99b913be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:07.482041 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aecb05-3746-44b2-a88f-c91d99b913be service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:07.482317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aecb05-3746-44b2-a88f-c91d99b913be service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:07.482587 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aecb05-3746-44b2-a88f-c91d99b913be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:07.492440 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aecb05-3746-44b2-a88f-c91d99b913be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:07.494142 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aecb05-3746-44b2-a88f-c91d99b913be service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:07.503668 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1296/2582] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:49:07.517484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1287/2583] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:49:07 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:49:07.521827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aecb05-3746-44b2-a88f-c91d99b913be service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:07.537547 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aecb05-3746-44b2-a88f-c91d99b913be service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:07.543922 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb5efb-af5a-49d3-991f-22e9198d83d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:07.544633 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb5efb-af5a-49d3-991f-22e9198d83d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:07.544984 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb5efb-af5a-49d3-991f-22e9198d83d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:07.545357 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb5efb-af5a-49d3-991f-22e9198d83d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:07.546072 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb5efb-af5a-49d3-991f-22e9198d83d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:07.547740 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb5efb-af5a-49d3-991f-22e9198d83d6 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:07.559252 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1297/2584] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:49:07.563400 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f033016-69ec-4c2d-b0cc-d71d9c16b94c admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:07.574454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f033016-69ec-4c2d-b0cc-d71d9c16b94c admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:07.579136 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5c5b4-face-4aee-bb20-d5c15a95a940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:07.579740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5c5b4-face-4aee-bb20-d5c15a95a940 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:07.580015 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5c5b4-face-4aee-bb20-d5c15a95a940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:07.580283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5c5b4-face-4aee-bb20-d5c15a95a940 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:07.580920 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b5c5b4-face-4aee-bb20-d5c15a95a940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:07.582538 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b5c5b4-face-4aee-bb20-d5c15a95a940 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:07.592938 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1288/2585] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:49:07.597064 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb5efb-af5a-49d3-991f-22e9198d83d6 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:07.607799 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb5efb-af5a-49d3-991f-22e9198d83d6 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:07.612672 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084a6a66-dcd1-464d-adf0-7da81442b633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:07.613380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a6a66-dcd1-464d-adf0-7da81442b633 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:07.613662 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a6a66-dcd1-464d-adf0-7da81442b633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:07.613936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a6a66-dcd1-464d-adf0-7da81442b633 admin admin] PATH_INFO: `/v3/projects/9ac2dfc942b9423a97d96353f595e5c0/users/879cdba023bc454db24418e04179b82f/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:07.625966 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:07.629871 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084a6a66-dcd1-464d-adf0-7da81442b633 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ac2dfc942b9423a97d96353f595e5c0, user_id=879cdba023bc454db24418e04179b82f, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:07.635274 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084a6a66-dcd1-464d-adf0-7da81442b633 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:07.729490 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1298/2586] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:49:07 2022] PUT /identity/v3/projects/9ac2dfc942b9423a97d96353f595e5c0/users/879cdba023bc454db24418e04179b82f/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:07.739831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa0e3d-d5ed-45ea-ab42-4be6530ba842 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:07.740122 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa0e3d-d5ed-45ea-ab42-4be6530ba842 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:07.740405 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa0e3d-d5ed-45ea-ab42-4be6530ba842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:07.773619 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-cffa0e3d-d5ed-45ea-ab42-4be6530ba842 None None] MFA Rules not processed for user `879cdba023bc454db24418e04179b82f`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:07.774981 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-cffa0e3d-d5ed-45ea-ab42-4be6530ba842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:07.842801 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1289/2587] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:07.852392 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084a6a66-dcd1-464d-adf0-7da81442b633 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:07.916445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084a6a66-dcd1-464d-adf0-7da81442b633 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:07.921381 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28080727-95de-4f8b-960a-38f1eeed6ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:07.922413 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28080727-95de-4f8b-960a-38f1eeed6ca2 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:07.922742 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28080727-95de-4f8b-960a-38f1eeed6ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:07.923011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28080727-95de-4f8b-960a-38f1eeed6ca2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:07.923627 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28080727-95de-4f8b-960a-38f1eeed6ca2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:07.925550 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28080727-95de-4f8b-960a-38f1eeed6ca2 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:07.952113 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa0e3d-d5ed-45ea-ab42-4be6530ba842 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:07.953694 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-cffa0e3d-d5ed-45ea-ab42-4be6530ba842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:07.968842 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa0e3d-d5ed-45ea-ab42-4be6530ba842 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:07.976436 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2bf886-c326-4a56-96dd-9301f69b569b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:07.976436 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bf886-c326-4a56-96dd-9301f69b569b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:07.976436 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bf886-c326-4a56-96dd-9301f69b569b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:07.976436 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bf886-c326-4a56-96dd-9301f69b569b admin admin] PATH_INFO: `/v3/projects/61508804d4294a11aff8825c1d25b657` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:07.977186 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2bf886-c326-4a56-96dd-9301f69b569b admin admin] RBAC: Authorizing `identity:delete_project(project_id=61508804d4294a11aff8825c1d25b657)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:07.979375 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2bf886-c326-4a56-96dd-9301f69b569b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:08.034630 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1299/2588] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:49:07 2022] POST /identity/v3/projects => generated 375 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:49:08.035686 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28080727-95de-4f8b-960a-38f1eeed6ca2 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:08.096091 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28080727-95de-4f8b-960a-38f1eeed6ca2 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:08.101127 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a465370-7b10-4414-b9d2-0deb7a1c6e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:08.101828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a465370-7b10-4414-b9d2-0deb7a1c6e2c admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:08.102111 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a465370-7b10-4414-b9d2-0deb7a1c6e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:08.102387 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a465370-7b10-4414-b9d2-0deb7a1c6e2c admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:08.103067 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a465370-7b10-4414-b9d2-0deb7a1c6e2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:08.105202 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a465370-7b10-4414-b9d2-0deb7a1c6e2c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:08.124480 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:08.134037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-6f2bf886-c326-4a56-96dd-9301f69b569b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61508804d4294a11aff8825c1d25b657', 'request_id': 'req-6f2bf886-c326-4a56-96dd-9301f69b569b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:08.140230 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-6f2bf886-c326-4a56-96dd-9301f69b569b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61508804d4294a11aff8825c1d25b657', 'request_id': 'req-6f2bf886-c326-4a56-96dd-9301f69b569b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:08.146156 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-6f2bf886-c326-4a56-96dd-9301f69b569b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61508804d4294a11aff8825c1d25b657', 'request_id': 'req-6f2bf886-c326-4a56-96dd-9301f69b569b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:08.242566 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-6f2bf886-c326-4a56-96dd-9301f69b569b admin admin] The token cache is being invalidate because project 61508804d4294a11aff8825c1d25b657 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:49:08.242850 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-6f2bf886-c326-4a56-96dd-9301f69b569b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:08.246194 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1300/2589] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:49:08 2022] POST /identity/v3/users => generated 294 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:49:08.246341 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-6f2bf886-c326-4a56-96dd-9301f69b569b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:08.246341 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1290/2590] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:49:07 2022] DELETE /identity/v3/projects/61508804d4294a11aff8825c1d25b657 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:49:08.247067 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a465370-7b10-4414-b9d2-0deb7a1c6e2c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:08.248916 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1a465370-7b10-4414-b9d2-0deb7a1c6e2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:08.249071 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2bf886-c326-4a56-96dd-9301f69b569b admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:08.250421 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6f2bf886-c326-4a56-96dd-9301f69b569b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:08.262056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a465370-7b10-4414-b9d2-0deb7a1c6e2c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:08.316939 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2bf886-c326-4a56-96dd-9301f69b569b admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:08.322680 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf915a0-3b03-4ce5-b165-cb76aa154b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:08.323211 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2d8c3-ef2f-44a2-a7ce-acf81f754561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:08.323338 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf915a0-3b03-4ce5-b165-cb76aa154b82 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:08.323617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf915a0-3b03-4ce5-b165-cb76aa154b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:08.323900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf915a0-3b03-4ce5-b165-cb76aa154b82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:08.324075 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2d8c3-ef2f-44a2-a7ce-acf81f754561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:08.324272 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2d8c3-ef2f-44a2-a7ce-acf81f754561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:08.324542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf915a0-3b03-4ce5-b165-cb76aa154b82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:08.324696 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2d8c3-ef2f-44a2-a7ce-acf81f754561 admin admin] PATH_INFO: `/v3/users/9acb36ac474449aa896879260d5bfccc` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:08.325843 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a2d8c3-ef2f-44a2-a7ce-acf81f754561 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9acb36ac474449aa896879260d5bfccc)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:08.326176 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf915a0-3b03-4ce5-b165-cb76aa154b82 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:08.327776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a2d8c3-ef2f-44a2-a7ce-acf81f754561 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:08.331393 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:08.336761 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1301/2591] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:49:08.340636 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf915a0-3b03-4ce5-b165-cb76aa154b82 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:08.350210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf915a0-3b03-4ce5-b165-cb76aa154b82 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:08.355235 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c8a63-e3ad-4a41-8f4c-90ec8c638b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:08.355824 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c8a63-e3ad-4a41-8f4c-90ec8c638b7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:08.356080 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c8a63-e3ad-4a41-8f4c-90ec8c638b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:08.356354 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c8a63-e3ad-4a41-8f4c-90ec8c638b7a admin admin] PATH_INFO: `/v3/projects/ff0854d3616747c18af8c8de206630c8/users/486a6618f43b4f09b70db66dc9513f3b/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:08.371318 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c8a63-e3ad-4a41-8f4c-90ec8c638b7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff0854d3616747c18af8c8de206630c8, user_id=486a6618f43b4f09b70db66dc9513f3b, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:08.374073 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c8a63-e3ad-4a41-8f4c-90ec8c638b7a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:08.476997 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1302/2592] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:49:08 2022] PUT /identity/v3/projects/ff0854d3616747c18af8c8de206630c8/users/486a6618f43b4f09b70db66dc9513f3b/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:49:08.481263 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c8a63-e3ad-4a41-8f4c-90ec8c638b7a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:08.513132 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-87a2d8c3-ef2f-44a2-a7ce-acf81f754561 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9acb36ac474449aa896879260d5bfccc'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:08.514169 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-87a2d8c3-ef2f-44a2-a7ce-acf81f754561 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9acb36ac474449aa896879260d5bfccc'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:08.515135 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-87a2d8c3-ef2f-44a2-a7ce-acf81f754561 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9acb36ac474449aa896879260d5bfccc'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:08.525345 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-87a2d8c3-ef2f-44a2-a7ce-acf81f754561 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9acb36ac474449aa896879260d5bfccc'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:08.557550 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c8a63-e3ad-4a41-8f4c-90ec8c638b7a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:08.562273 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d5dd5-74b9-4fe0-919a-05a97d7d9528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:08.562871 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d5dd5-74b9-4fe0-919a-05a97d7d9528 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:08.563150 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d5dd5-74b9-4fe0-919a-05a97d7d9528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:08.563424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d5dd5-74b9-4fe0-919a-05a97d7d9528 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:08.564066 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d5dd5-74b9-4fe0-919a-05a97d7d9528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:08.565697 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d5dd5-74b9-4fe0-919a-05a97d7d9528 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:08.575437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1303/2593] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:49:08.579642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d5dd5-74b9-4fe0-919a-05a97d7d9528 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:08.583809 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a61d5dd5-74b9-4fe0-919a-05a97d7d9528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:08.593701 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d5dd5-74b9-4fe0-919a-05a97d7d9528 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:08.598927 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73807d92-0291-4a83-9d82-4059c4eecf43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:08.599561 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73807d92-0291-4a83-9d82-4059c4eecf43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:08.599836 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73807d92-0291-4a83-9d82-4059c4eecf43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:08.600108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73807d92-0291-4a83-9d82-4059c4eecf43 admin admin] PATH_INFO: `/v3/projects/ff0854d3616747c18af8c8de206630c8/users/486a6618f43b4f09b70db66dc9513f3b/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:08.602573 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73807d92-0291-4a83-9d82-4059c4eecf43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff0854d3616747c18af8c8de206630c8, user_id=486a6618f43b4f09b70db66dc9513f3b, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:08.605519 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73807d92-0291-4a83-9d82-4059c4eecf43 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:08.613801 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-87a2d8c3-ef2f-44a2-a7ce-acf81f754561 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9acb36ac474449aa896879260d5bfccc'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:08.615299 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1304/2594] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:49:08 2022] PUT /identity/v3/projects/ff0854d3616747c18af8c8de206630c8/users/486a6618f43b4f09b70db66dc9513f3b/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:49:08.626756 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d70215-9980-4e25-95a2-6e7bea6d6314 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:08.627041 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d70215-9980-4e25-95a2-6e7bea6d6314 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:08.627313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d70215-9980-4e25-95a2-6e7bea6d6314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:08.636834 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1291/2595] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:49:08 2022] DELETE /identity/v3/users/9acb36ac474449aa896879260d5bfccc => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:49:08.655754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-47d70215-9980-4e25-95a2-6e7bea6d6314 None None] MFA Rules not processed for user `486a6618f43b4f09b70db66dc9513f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:08.656990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-47d70215-9980-4e25-95a2-6e7bea6d6314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:08.730285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1305/2596] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:08 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:49:08.742919 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2d8c3-ef2f-44a2-a7ce-acf81f754561 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:08.744416 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-87a2d8c3-ef2f-44a2-a7ce-acf81f754561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:08.792300 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:08.805323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2d8c3-ef2f-44a2-a7ce-acf81f754561 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:08.809463 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb88370-40dc-4b0f-9b4e-ad99360c021c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:08.810282 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb88370-40dc-4b0f-9b4e-ad99360c021c service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:08.810472 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb88370-40dc-4b0f-9b4e-ad99360c021c service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:08.810788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb88370-40dc-4b0f-9b4e-ad99360c021c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:08.812542 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-abb88370-40dc-4b0f-9b4e-ad99360c021c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:08.877291 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb88370-40dc-4b0f-9b4e-ad99360c021c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:08.878879 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb88370-40dc-4b0f-9b4e-ad99360c021c service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:08.888909 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1292/2597] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:49:09.053074 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d70215-9980-4e25-95a2-6e7bea6d6314 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:09.053074 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-47d70215-9980-4e25-95a2-6e7bea6d6314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:09.108191 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:09.127907 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d70215-9980-4e25-95a2-6e7bea6d6314 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:09.133030 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d933b4c-fe8e-4fd1-aa0b-d278eb4f3b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:09.133697 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d933b4c-fe8e-4fd1-aa0b-d278eb4f3b16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:09.133980 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d933b4c-fe8e-4fd1-aa0b-d278eb4f3b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:09.134255 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d933b4c-fe8e-4fd1-aa0b-d278eb4f3b16 admin admin] PATH_INFO: `/v3/projects/decf39b61a19480dba5e4fd3c4f62e1d` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:09.135382 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d933b4c-fe8e-4fd1-aa0b-d278eb4f3b16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=decf39b61a19480dba5e4fd3c4f62e1d)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:09.137329 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d933b4c-fe8e-4fd1-aa0b-d278eb4f3b16 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:09.305042 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-4d933b4c-fe8e-4fd1-aa0b-d278eb4f3b16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'decf39b61a19480dba5e4fd3c4f62e1d', 'request_id': 'req-4d933b4c-fe8e-4fd1-aa0b-d278eb4f3b16'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:09.396715 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-4d933b4c-fe8e-4fd1-aa0b-d278eb4f3b16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'decf39b61a19480dba5e4fd3c4f62e1d', 'request_id': 'req-4d933b4c-fe8e-4fd1-aa0b-d278eb4f3b16'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:09.403702 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-4d933b4c-fe8e-4fd1-aa0b-d278eb4f3b16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'decf39b61a19480dba5e4fd3c4f62e1d', 'request_id': 'req-4d933b4c-fe8e-4fd1-aa0b-d278eb4f3b16'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:09.418380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-4d933b4c-fe8e-4fd1-aa0b-d278eb4f3b16 admin admin] The token cache is being invalidate because project decf39b61a19480dba5e4fd3c4f62e1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:49:09.418731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-4d933b4c-fe8e-4fd1-aa0b-d278eb4f3b16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:09.419380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-4d933b4c-fe8e-4fd1-aa0b-d278eb4f3b16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:09.421231 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1306/2598] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:49:09 2022] DELETE /identity/v3/projects/decf39b61a19480dba5e4fd3c4f62e1d => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:49:09.426700 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ad3e0-1802-431d-95fb-cf0bd6aa4444 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:09.427083 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ad3e0-1802-431d-95fb-cf0bd6aa4444 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:09.427331 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ad3e0-1802-431d-95fb-cf0bd6aa4444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:09.449967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-b70ad3e0-1802-431d-95fb-cf0bd6aa4444 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:09.451246 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b70ad3e0-1802-431d-95fb-cf0bd6aa4444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:09.497991 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d933b4c-fe8e-4fd1-aa0b-d278eb4f3b16 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:09.499451 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4d933b4c-fe8e-4fd1-aa0b-d278eb4f3b16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:09.538776 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1293/2599] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:09.548044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28498b24-33d4-4d2e-b204-f4ced6f92961 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:09.548334 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28498b24-33d4-4d2e-b204-f4ced6f92961 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:09.548603 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28498b24-33d4-4d2e-b204-f4ced6f92961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:09.573566 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-28498b24-33d4-4d2e-b204-f4ced6f92961 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:09.574769 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-28498b24-33d4-4d2e-b204-f4ced6f92961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:09.582727 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1294/2600] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:49:09.582866 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d933b4c-fe8e-4fd1-aa0b-d278eb4f3b16 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:09.587355 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28498b24-33d4-4d2e-b204-f4ced6f92961 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:09.589326 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35936770-ec39-4c3e-83ff-77f326dd09aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:09.589819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35936770-ec39-4c3e-83ff-77f326dd09aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:09.591012 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35936770-ec39-4c3e-83ff-77f326dd09aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:09.591366 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35936770-ec39-4c3e-83ff-77f326dd09aa admin admin] PATH_INFO: `/v3/users/b1af67c1a77e42cdb0652eb39b2a7e57` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:09.593011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35936770-ec39-4c3e-83ff-77f326dd09aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1af67c1a77e42cdb0652eb39b2a7e57)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:09.595169 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35936770-ec39-4c3e-83ff-77f326dd09aa admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:09.617501 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28498b24-33d4-4d2e-b204-f4ced6f92961 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:09.622541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3cc17-082a-41fe-9060-df7c5184d369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:09.623359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3cc17-082a-41fe-9060-df7c5184d369 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:09.623626 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3cc17-082a-41fe-9060-df7c5184d369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:09.623904 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3cc17-082a-41fe-9060-df7c5184d369 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:09.624668 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3cc17-082a-41fe-9060-df7c5184d369 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:49:09.624972 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3cc17-082a-41fe-9060-df7c5184d369 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:09.626607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3cc17-082a-41fe-9060-df7c5184d369 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:09.640778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1295/2601] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:49:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:49:09.644974 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3cc17-082a-41fe-9060-df7c5184d369 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:09.655837 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3cc17-082a-41fe-9060-df7c5184d369 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:09.660645 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5dbae-cfde-49f8-b63b-833e81b26a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:09.661277 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5dbae-cfde-49f8-b63b-833e81b26a11 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:09.661561 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5dbae-cfde-49f8-b63b-833e81b26a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:09.661834 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5dbae-cfde-49f8-b63b-833e81b26a11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:09.662481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5dbae-cfde-49f8-b63b-833e81b26a11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:09.664684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5dbae-cfde-49f8-b63b-833e81b26a11 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:09.806338 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1296/2602] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:49:09 2022] POST /identity/v3/projects => generated 389 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:49:09.807329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5dbae-cfde-49f8-b63b-833e81b26a11 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:09.808933 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-67b5dbae-cfde-49f8-b63b-833e81b26a11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:09.834980 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-35936770-ec39-4c3e-83ff-77f326dd09aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1af67c1a77e42cdb0652eb39b2a7e57'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:09.835779 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-35936770-ec39-4c3e-83ff-77f326dd09aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1af67c1a77e42cdb0652eb39b2a7e57'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:09.836420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-35936770-ec39-4c3e-83ff-77f326dd09aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1af67c1a77e42cdb0652eb39b2a7e57'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:09.843545 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-35936770-ec39-4c3e-83ff-77f326dd09aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1af67c1a77e42cdb0652eb39b2a7e57'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:09.869908 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:09.875003 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5dbae-cfde-49f8-b63b-833e81b26a11 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:09.879627 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0f352-005f-4d27-995d-682f0bc4d03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:09.883870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0f352-005f-4d27-995d-682f0bc4d03e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:09.883870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0f352-005f-4d27-995d-682f0bc4d03e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:09.883870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0f352-005f-4d27-995d-682f0bc4d03e admin admin] PATH_INFO: `/v3/users/a02c4fdcb5ad418d93e0629f604babea` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:09.883870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab0f352-005f-4d27-995d-682f0bc4d03e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a02c4fdcb5ad418d93e0629f604babea)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:09.884393 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab0f352-005f-4d27-995d-682f0bc4d03e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:09.910792 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-35936770-ec39-4c3e-83ff-77f326dd09aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1af67c1a77e42cdb0652eb39b2a7e57'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:09.921589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:09.929571 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1307/2603] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:49:09 2022] DELETE /identity/v3/users/b1af67c1a77e42cdb0652eb39b2a7e57 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:09.931033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35936770-ec39-4c3e-83ff-77f326dd09aa admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:09.932351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-35936770-ec39-4c3e-83ff-77f326dd09aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:09.995097 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35936770-ec39-4c3e-83ff-77f326dd09aa admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:09.999712 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df4dbe-24d6-4952-a7e4-c2f2b7b4e7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:10.000311 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df4dbe-24d6-4952-a7e4-c2f2b7b4e7f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:10.000652 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df4dbe-24d6-4952-a7e4-c2f2b7b4e7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:10.000925 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df4dbe-24d6-4952-a7e4-c2f2b7b4e7f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:10.001573 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97df4dbe-24d6-4952-a7e4-c2f2b7b4e7f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:10.003479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97df4dbe-24d6-4952-a7e4-c2f2b7b4e7f5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:10.108907 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1308/2604] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:49:09 2022] POST /identity/v3/users => generated 295 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:49:10.108907 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5784c-4d2b-453f-aabb-432a730bfd64 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:10.108907 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5784c-4d2b-453f-aabb-432a730bfd64 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:10.108907 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5784c-4d2b-453f-aabb-432a730bfd64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:10.119908 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-0ab0f352-005f-4d27-995d-682f0bc4d03e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a02c4fdcb5ad418d93e0629f604babea'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:10.120579 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-0ab0f352-005f-4d27-995d-682f0bc4d03e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a02c4fdcb5ad418d93e0629f604babea'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:10.121216 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-0ab0f352-005f-4d27-995d-682f0bc4d03e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a02c4fdcb5ad418d93e0629f604babea'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:10.128261 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-0ab0f352-005f-4d27-995d-682f0bc4d03e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a02c4fdcb5ad418d93e0629f604babea'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:10.129661 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4ff5784c-4d2b-453f-aabb-432a730bfd64 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:10.131035 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4ff5784c-4d2b-453f-aabb-432a730bfd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:10.139330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1309/2605] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:10.140265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5784c-4d2b-453f-aabb-432a730bfd64 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:10.141535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4ff5784c-4d2b-453f-aabb-432a730bfd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:10.153221 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5784c-4d2b-453f-aabb-432a730bfd64 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:10.157979 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a6f24-c52d-4e2f-9f00-6aebf2c33561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:10.159816 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a6f24-c52d-4e2f-9f00-6aebf2c33561 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:10.159816 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a6f24-c52d-4e2f-9f00-6aebf2c33561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:10.159816 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a6f24-c52d-4e2f-9f00-6aebf2c33561 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:10.160104 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a6f24-c52d-4e2f-9f00-6aebf2c33561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:10.161651 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a6f24-c52d-4e2f-9f00-6aebf2c33561 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:10.172351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1310/2606] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:49:10.173228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a6f24-c52d-4e2f-9f00-6aebf2c33561 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:10.175039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8e3a6f24-c52d-4e2f-9f00-6aebf2c33561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:10.231584 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a6f24-c52d-4e2f-9f00-6aebf2c33561 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:10.235290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835fdbc-2af7-45c6-ba5a-c418ca1b63dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:10.235911 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835fdbc-2af7-45c6-ba5a-c418ca1b63dc service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:10.236334 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835fdbc-2af7-45c6-ba5a-c418ca1b63dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:10.236458 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835fdbc-2af7-45c6-ba5a-c418ca1b63dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:10.237051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-0ab0f352-005f-4d27-995d-682f0bc4d03e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a02c4fdcb5ad418d93e0629f604babea'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:10.246483 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835fdbc-2af7-45c6-ba5a-c418ca1b63dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:10.248543 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835fdbc-2af7-45c6-ba5a-c418ca1b63dc service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:10.255268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1297/2607] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:49:09 2022] DELETE /identity/v3/users/a02c4fdcb5ad418d93e0629f604babea => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:10.258520 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0f352-005f-4d27-995d-682f0bc4d03e admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:10.262056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1311/2608] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:49:10.325800 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0f352-005f-4d27-995d-682f0bc4d03e admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:10.330404 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a681b9-2ff5-4de7-81dd-437468dae7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:10.331011 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a681b9-2ff5-4de7-81dd-437468dae7d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:10.331332 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a681b9-2ff5-4de7-81dd-437468dae7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:10.331587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a681b9-2ff5-4de7-81dd-437468dae7d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:10.332229 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a681b9-2ff5-4de7-81dd-437468dae7d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:10.333890 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a681b9-2ff5-4de7-81dd-437468dae7d7 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:10.344470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1298/2609] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:49:10.353376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835fdbc-2af7-45c6-ba5a-c418ca1b63dc service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:10.369072 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835fdbc-2af7-45c6-ba5a-c418ca1b63dc service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:10.375841 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d975b-1395-44cb-be03-16e06050cd6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:10.377042 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d975b-1395-44cb-be03-16e06050cd6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:10.377540 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d975b-1395-44cb-be03-16e06050cd6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:10.378019 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d975b-1395-44cb-be03-16e06050cd6d admin admin] PATH_INFO: `/v3/projects/7906051e49054eab940ef557e6da267e/users/d4ac39859e8c490182c99a468dd92011/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:10.395720 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d975b-1395-44cb-be03-16e06050cd6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7906051e49054eab940ef557e6da267e, user_id=d4ac39859e8c490182c99a468dd92011, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:10.399044 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d975b-1395-44cb-be03-16e06050cd6d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:10.536811 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1312/2610] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:49:10 2022] PUT /identity/v3/projects/7906051e49054eab940ef557e6da267e/users/d4ac39859e8c490182c99a468dd92011/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:10.546693 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a681b9-2ff5-4de7-81dd-437468dae7d7 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:10.548550 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769f59e-0e7e-490e-a933-b87b18037ba0 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:10.548890 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769f59e-0e7e-490e-a933-b87b18037ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:10.549089 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769f59e-0e7e-490e-a933-b87b18037ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:10.551080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-57a681b9-2ff5-4de7-81dd-437468dae7d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:10.572190 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a681b9-2ff5-4de7-81dd-437468dae7d7 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:10.596598 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0769f59e-0e7e-490e-a933-b87b18037ba0 None None] MFA Rules not processed for user `d4ac39859e8c490182c99a468dd92011`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:10.598121 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0769f59e-0e7e-490e-a933-b87b18037ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:10.640333 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b82515-110a-4acd-aa32-74b690906b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:10.641405 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b82515-110a-4acd-aa32-74b690906b1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:10.641726 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b82515-110a-4acd-aa32-74b690906b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:10.642007 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b82515-110a-4acd-aa32-74b690906b1a admin admin] PATH_INFO: `/v3/projects/9537e903aa1646ee8f3774b7dd417309` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:10.643236 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b82515-110a-4acd-aa32-74b690906b1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9537e903aa1646ee8f3774b7dd417309)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:10.645201 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b82515-110a-4acd-aa32-74b690906b1a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:10.660057 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:10.675813 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1313/2611] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:10 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:10.679667 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769f59e-0e7e-490e-a933-b87b18037ba0 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:10.690433 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769f59e-0e7e-490e-a933-b87b18037ba0 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:10.694880 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:10.698069 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084b7a5-2f3d-48d7-bca0-e6ba2ff2fe9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:10.698924 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084b7a5-2f3d-48d7-bca0-e6ba2ff2fe9a admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:10.699199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084b7a5-2f3d-48d7-bca0-e6ba2ff2fe9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:10.699488 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084b7a5-2f3d-48d7-bca0-e6ba2ff2fe9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:10.700117 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6084b7a5-2f3d-48d7-bca0-e6ba2ff2fe9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:10.702075 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6084b7a5-2f3d-48d7-bca0-e6ba2ff2fe9a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:10.798788 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1314/2612] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:49:10 2022] POST /identity/v3/projects => generated 389 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:49:10.802727 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084b7a5-2f3d-48d7-bca0-e6ba2ff2fe9a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:10.805303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6084b7a5-2f3d-48d7-bca0-e6ba2ff2fe9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:10.815392 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-81b82515-110a-4acd-aa32-74b690906b1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9537e903aa1646ee8f3774b7dd417309', 'request_id': 'req-81b82515-110a-4acd-aa32-74b690906b1a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:10.822089 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-81b82515-110a-4acd-aa32-74b690906b1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9537e903aa1646ee8f3774b7dd417309', 'request_id': 'req-81b82515-110a-4acd-aa32-74b690906b1a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:10.829429 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-81b82515-110a-4acd-aa32-74b690906b1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9537e903aa1646ee8f3774b7dd417309', 'request_id': 'req-81b82515-110a-4acd-aa32-74b690906b1a'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:10.880296 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084b7a5-2f3d-48d7-bca0-e6ba2ff2fe9a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:10.889276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293beff-8c64-49e7-ac36-c8fdf26fa870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:10.889276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293beff-8c64-49e7-ac36-c8fdf26fa870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:10.889276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293beff-8c64-49e7-ac36-c8fdf26fa870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:10.889276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293beff-8c64-49e7-ac36-c8fdf26fa870 admin admin] PATH_INFO: `/v3/projects/3597de62832549adab0206d0ba797e74` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:10.889784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-81b82515-110a-4acd-aa32-74b690906b1a admin admin] The token cache is being invalidate because project 9537e903aa1646ee8f3774b7dd417309 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:49:10.889784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-81b82515-110a-4acd-aa32-74b690906b1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:10.889784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-81b82515-110a-4acd-aa32-74b690906b1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:10.890687 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1299/2613] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:49:10 2022] DELETE /identity/v3/projects/9537e903aa1646ee8f3774b7dd417309 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:10.890807 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293beff-8c64-49e7-ac36-c8fdf26fa870 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3597de62832549adab0206d0ba797e74)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:10.891529 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b82515-110a-4acd-aa32-74b690906b1a admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:10.893006 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-81b82515-110a-4acd-aa32-74b690906b1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:10.896153 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293beff-8c64-49e7-ac36-c8fdf26fa870 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:10.963281 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b82515-110a-4acd-aa32-74b690906b1a admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:10.968017 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdcb50e-94f7-4f74-9689-2acb8c7c2a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:10.968622 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdcb50e-94f7-4f74-9689-2acb8c7c2a54 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:10.968901 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdcb50e-94f7-4f74-9689-2acb8c7c2a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:10.969198 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdcb50e-94f7-4f74-9689-2acb8c7c2a54 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:10.969818 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdcb50e-94f7-4f74-9689-2acb8c7c2a54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:10.971869 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdcb50e-94f7-4f74-9689-2acb8c7c2a54 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:10.986155 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f293beff-8c64-49e7-ac36-c8fdf26fa870 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3597de62832549adab0206d0ba797e74', 'request_id': 'req-f293beff-8c64-49e7-ac36-c8fdf26fa870'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:11.048616 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f293beff-8c64-49e7-ac36-c8fdf26fa870 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3597de62832549adab0206d0ba797e74', 'request_id': 'req-f293beff-8c64-49e7-ac36-c8fdf26fa870'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:11.056370 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1300/2614] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:49:10 2022] POST /identity/v3/users => generated 301 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:49:11.056765 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f293beff-8c64-49e7-ac36-c8fdf26fa870 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3597de62832549adab0206d0ba797e74', 'request_id': 'req-f293beff-8c64-49e7-ac36-c8fdf26fa870'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:11.057957 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db01088-da7d-4be8-a770-b462e88a80c4 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:11.058247 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db01088-da7d-4be8-a770-b462e88a80c4 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:11.058664 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db01088-da7d-4be8-a770-b462e88a80c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:11.068075 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f293beff-8c64-49e7-ac36-c8fdf26fa870 admin admin] The token cache is being invalidate because project 3597de62832549adab0206d0ba797e74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:49:11.068362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f293beff-8c64-49e7-ac36-c8fdf26fa870 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:11.069076 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f293beff-8c64-49e7-ac36-c8fdf26fa870 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:11.071139 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1315/2615] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:49:10 2022] DELETE /identity/v3/projects/3597de62832549adab0206d0ba797e74 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:11.071972 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293beff-8c64-49e7-ac36-c8fdf26fa870 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:11.073370 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f293beff-8c64-49e7-ac36-c8fdf26fa870 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:11.079176 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-8db01088-da7d-4be8-a770-b462e88a80c4 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:11.080367 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8db01088-da7d-4be8-a770-b462e88a80c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:11.122382 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:11.133993 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293beff-8c64-49e7-ac36-c8fdf26fa870 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:11.137671 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2228fa-399d-4e72-82ab-c0ab4d743f4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:11.138283 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2228fa-399d-4e72-82ab-c0ab4d743f4c service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:11.138564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2228fa-399d-4e72-82ab-c0ab4d743f4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:11.138837 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2228fa-399d-4e72-82ab-c0ab4d743f4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:11.140617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-da2228fa-399d-4e72-82ab-c0ab4d743f4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:11.142180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1301/2616] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:11.150069 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db01088-da7d-4be8-a770-b462e88a80c4 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:11.151375 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8db01088-da7d-4be8-a770-b462e88a80c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:11.163727 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db01088-da7d-4be8-a770-b462e88a80c4 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:11.168679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94932ac6-a3bd-49cc-9d37-50f4bce98c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:11.169299 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94932ac6-a3bd-49cc-9d37-50f4bce98c6e admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:11.169578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94932ac6-a3bd-49cc-9d37-50f4bce98c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:11.169845 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94932ac6-a3bd-49cc-9d37-50f4bce98c6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:11.170477 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94932ac6-a3bd-49cc-9d37-50f4bce98c6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:11.172141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94932ac6-a3bd-49cc-9d37-50f4bce98c6e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:11.182206 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1302/2617] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:49:11.183078 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94932ac6-a3bd-49cc-9d37-50f4bce98c6e admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:11.184960 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-94932ac6-a3bd-49cc-9d37-50f4bce98c6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:11.198198 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94932ac6-a3bd-49cc-9d37-50f4bce98c6e admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:11.201442 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2228fa-399d-4e72-82ab-c0ab4d743f4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:11.202907 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2228fa-399d-4e72-82ab-c0ab4d743f4c service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:11.203132 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ee896-53c9-4349-a4ee-c01968ed5b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:11.203830 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ee896-53c9-4349-a4ee-c01968ed5b5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:11.204219 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ee896-53c9-4349-a4ee-c01968ed5b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:11.204586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ee896-53c9-4349-a4ee-c01968ed5b5a admin admin] PATH_INFO: `/v3/users/537bcc92d2b34c1c865708fa74c69bf3` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:11.205835 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ee896-53c9-4349-a4ee-c01968ed5b5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=537bcc92d2b34c1c865708fa74c69bf3)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:11.207794 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ee896-53c9-4349-a4ee-c01968ed5b5a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:11.212742 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1316/2618] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:49:11.218279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383da56-eb56-4477-86a5-63381ba8c42e None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:11.218507 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383da56-eb56-4477-86a5-63381ba8c42e None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:11.218781 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383da56-eb56-4477-86a5-63381ba8c42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:11.223209 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=88,keystone:DELETE=8,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:11.240293 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0383da56-eb56-4477-86a5-63381ba8c42e None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:11.242286 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0383da56-eb56-4477-86a5-63381ba8c42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:11.251186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1317/2619] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:11.251972 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383da56-eb56-4477-86a5-63381ba8c42e None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:11.261793 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383da56-eb56-4477-86a5-63381ba8c42e None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:11.266538 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ef761-448e-4b8b-9c4a-a290c3f28785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:11.267186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ef761-448e-4b8b-9c4a-a290c3f28785 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:11.269323 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ef761-448e-4b8b-9c4a-a290c3f28785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:11.269323 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ef761-448e-4b8b-9c4a-a290c3f28785 admin admin] PATH_INFO: `/v3/projects/7b1f2f9c73a34fae9c538aa64ef014c3/users/3f7c1442bb8b457da6be54d2f1dc1eb6/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:11.285086 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ef761-448e-4b8b-9c4a-a290c3f28785 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b1f2f9c73a34fae9c538aa64ef014c3, user_id=3f7c1442bb8b457da6be54d2f1dc1eb6, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:11.287939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ef761-448e-4b8b-9c4a-a290c3f28785 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:11.402044 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1318/2620] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:49:11 2022] PUT /identity/v3/projects/7b1f2f9c73a34fae9c538aa64ef014c3/users/3f7c1442bb8b457da6be54d2f1dc1eb6/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:11.403030 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ef761-448e-4b8b-9c4a-a290c3f28785 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:11.423781 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-7c7ee896-53c9-4349-a4ee-c01968ed5b5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '537bcc92d2b34c1c865708fa74c69bf3'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:11.424873 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-7c7ee896-53c9-4349-a4ee-c01968ed5b5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '537bcc92d2b34c1c865708fa74c69bf3'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:11.426135 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-7c7ee896-53c9-4349-a4ee-c01968ed5b5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '537bcc92d2b34c1c865708fa74c69bf3'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:11.434180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-7c7ee896-53c9-4349-a4ee-c01968ed5b5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '537bcc92d2b34c1c865708fa74c69bf3'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:11.467579 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ef761-448e-4b8b-9c4a-a290c3f28785 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:11.472354 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c16443-e610-4b81-b848-4779c13ca8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:11.472890 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c16443-e610-4b81-b848-4779c13ca8d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:11.473196 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c16443-e610-4b81-b848-4779c13ca8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:11.473470 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c16443-e610-4b81-b848-4779c13ca8d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:11.474104 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c16443-e610-4b81-b848-4779c13ca8d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:49:11.474402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c16443-e610-4b81-b848-4779c13ca8d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:11.476080 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c16443-e610-4b81-b848-4779c13ca8d5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:11.487471 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1319/2621] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:49:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:49:11.488319 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c16443-e610-4b81-b848-4779c13ca8d5 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:11.490219 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f3c16443-e610-4b81-b848-4779c13ca8d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:11.503522 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c16443-e610-4b81-b848-4779c13ca8d5 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:11.509926 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f70e8-a9ae-4d58-9a4f-210e1f7f8822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:11.510506 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f70e8-a9ae-4d58-9a4f-210e1f7f8822 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:11.510787 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f70e8-a9ae-4d58-9a4f-210e1f7f8822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:11.511062 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f70e8-a9ae-4d58-9a4f-210e1f7f8822 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:11.511699 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f70e8-a9ae-4d58-9a4f-210e1f7f8822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:11.513330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f70e8-a9ae-4d58-9a4f-210e1f7f8822 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:11.521826 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-7c7ee896-53c9-4349-a4ee-c01968ed5b5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '537bcc92d2b34c1c865708fa74c69bf3'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:11.530810 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1320/2622] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:49:11.530810 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f70e8-a9ae-4d58-9a4f-210e1f7f8822 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:11.530810 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d17f70e8-a9ae-4d58-9a4f-210e1f7f8822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:11.539158 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f70e8-a9ae-4d58-9a4f-210e1f7f8822 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:11.543770 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f88c6c-4fa3-4330-a35e-af669ea266b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:11.544428 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1303/2623] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:49:11 2022] DELETE /identity/v3/users/537bcc92d2b34c1c865708fa74c69bf3 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:11.544563 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f88c6c-4fa3-4330-a35e-af669ea266b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:11.544662 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f88c6c-4fa3-4330-a35e-af669ea266b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:11.544885 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f88c6c-4fa3-4330-a35e-af669ea266b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:11.545247 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ee896-53c9-4349-a4ee-c01968ed5b5a admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:11.545600 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f88c6c-4fa3-4330-a35e-af669ea266b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:11.547426 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f88c6c-4fa3-4330-a35e-af669ea266b6 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:11.609124 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ee896-53c9-4349-a4ee-c01968ed5b5a admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:11.614031 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580d6c6-df2e-422c-8acf-8281079ad82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:11.617121 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580d6c6-df2e-422c-8acf-8281079ad82b admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:11.617121 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580d6c6-df2e-422c-8acf-8281079ad82b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:11.617121 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580d6c6-df2e-422c-8acf-8281079ad82b admin admin] PATH_INFO: `/v3/projects/7b1f2f9c73a34fae9c538aa64ef014c3/users/3f7c1442bb8b457da6be54d2f1dc1eb6/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:11.617517 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3580d6c6-df2e-422c-8acf-8281079ad82b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b1f2f9c73a34fae9c538aa64ef014c3, user_id=3f7c1442bb8b457da6be54d2f1dc1eb6, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:11.620199 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3580d6c6-df2e-422c-8acf-8281079ad82b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:11.634743 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1304/2624] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:49:11 2022] PUT /identity/v3/projects/7b1f2f9c73a34fae9c538aa64ef014c3/users/3f7c1442bb8b457da6be54d2f1dc1eb6/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:11.645124 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1321/2625] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:49:11 2022] POST /identity/v3/projects => generated 371 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:49:11.647754 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723aa3a-a4b1-4bd5-bef8-3427e9e1b38a None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:11.647754 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723aa3a-a4b1-4bd5-bef8-3427e9e1b38a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:11.647754 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723aa3a-a4b1-4bd5-bef8-3427e9e1b38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:11.649232 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f88c6c-4fa3-4330-a35e-af669ea266b6 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:11.675703 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-2723aa3a-a4b1-4bd5-bef8-3427e9e1b38a None None] MFA Rules not processed for user `3f7c1442bb8b457da6be54d2f1dc1eb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:11.676992 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2723aa3a-a4b1-4bd5-bef8-3427e9e1b38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:11.686206 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:11.718947 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f88c6c-4fa3-4330-a35e-af669ea266b6 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:11.724257 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5d542-2bc8-4b04-b2f3-33fb10846263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:11.724887 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5d542-2bc8-4b04-b2f3-33fb10846263 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:11.725294 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5d542-2bc8-4b04-b2f3-33fb10846263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:11.725640 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5d542-2bc8-4b04-b2f3-33fb10846263 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:11.726348 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5d542-2bc8-4b04-b2f3-33fb10846263 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:11.728337 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5d542-2bc8-4b04-b2f3-33fb10846263 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:11.751443 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1305/2626] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:11 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:11.762886 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2723aa3a-a4b1-4bd5-bef8-3427e9e1b38a None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:11.764417 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2723aa3a-a4b1-4bd5-bef8-3427e9e1b38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:11.782117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:11.819843 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1322/2627] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:49:11 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:49:11.823799 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5d542-2bc8-4b04-b2f3-33fb10846263 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:11.828083 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2723aa3a-a4b1-4bd5-bef8-3427e9e1b38a None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:11.832317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613cb95b-3662-47b0-98ed-9107896dcca4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:11.832996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613cb95b-3662-47b0-98ed-9107896dcca4 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:11.833399 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613cb95b-3662-47b0-98ed-9107896dcca4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:11.833757 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613cb95b-3662-47b0-98ed-9107896dcca4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:11.836207 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5d542-2bc8-4b04-b2f3-33fb10846263 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:11.843075 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27db045-d322-458e-91bc-aacee82eb616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:11.843330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27db045-d322-458e-91bc-aacee82eb616 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:11.843576 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27db045-d322-458e-91bc-aacee82eb616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:11.844442 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27db045-d322-458e-91bc-aacee82eb616 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:11.844569 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27db045-d322-458e-91bc-aacee82eb616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:11.845775 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613cb95b-3662-47b0-98ed-9107896dcca4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:11.846113 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27db045-d322-458e-91bc-aacee82eb616 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:11.847288 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613cb95b-3662-47b0-98ed-9107896dcca4 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:11.858171 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1306/2628] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:11 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:49:11.858679 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1323/2629] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:49:11.862480 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613cb95b-3662-47b0-98ed-9107896dcca4 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:11.873845 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613cb95b-3662-47b0-98ed-9107896dcca4 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:11.879481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf62c5e-2baa-4a3f-9e6a-c339403232fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:11.880172 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf62c5e-2baa-4a3f-9e6a-c339403232fc admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:11.880461 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf62c5e-2baa-4a3f-9e6a-c339403232fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:11.880729 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf62c5e-2baa-4a3f-9e6a-c339403232fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:11.881401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf62c5e-2baa-4a3f-9e6a-c339403232fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:11.884409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf62c5e-2baa-4a3f-9e6a-c339403232fc admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:11.896470 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1307/2630] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:49:11.907635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27db045-d322-458e-91bc-aacee82eb616 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:11.918542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27db045-d322-458e-91bc-aacee82eb616 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:11.923987 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974f496-0d1b-43f0-be1f-c98ed7f4c4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:11.924740 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974f496-0d1b-43f0-be1f-c98ed7f4c4cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:11.925106 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974f496-0d1b-43f0-be1f-c98ed7f4c4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:11.925416 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974f496-0d1b-43f0-be1f-c98ed7f4c4cc admin admin] PATH_INFO: `/v3/projects/39502bc219be4bb1bbeb04b0fcde1ca2/users/587a257431cc44e098607ac45eead615/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:11.943464 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974f496-0d1b-43f0-be1f-c98ed7f4c4cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=39502bc219be4bb1bbeb04b0fcde1ca2, user_id=587a257431cc44e098607ac45eead615, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:11.946271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974f496-0d1b-43f0-be1f-c98ed7f4c4cc admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:11.992896 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf62c5e-2baa-4a3f-9e6a-c339403232fc admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:11.994895 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-7bf62c5e-2baa-4a3f-9e6a-c339403232fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:12.010201 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf62c5e-2baa-4a3f-9e6a-c339403232fc admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:12.023787 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce736a31-78a6-4760-a8a1-5a10ac2e7b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:12.023787 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce736a31-78a6-4760-a8a1-5a10ac2e7b86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:12.024583 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce736a31-78a6-4760-a8a1-5a10ac2e7b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:12.024583 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce736a31-78a6-4760-a8a1-5a10ac2e7b86 admin admin] PATH_INFO: `/v3/projects/4e510e9cdaa4457982ce1bc7af9aeee1` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:12.027059 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce736a31-78a6-4760-a8a1-5a10ac2e7b86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e510e9cdaa4457982ce1bc7af9aeee1)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:12.028965 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce736a31-78a6-4760-a8a1-5a10ac2e7b86 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:12.082371 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1324/2631] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:49:11 2022] PUT /identity/v3/projects/39502bc219be4bb1bbeb04b0fcde1ca2/users/587a257431cc44e098607ac45eead615/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:12.083211 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974f496-0d1b-43f0-be1f-c98ed7f4c4cc admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:12.146092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974f496-0d1b-43f0-be1f-c98ed7f4c4cc admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:12.151376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4539f52-30ec-4bd5-b875-8e57afd3a44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:12.152000 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4539f52-30ec-4bd5-b875-8e57afd3a44d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:12.152277 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4539f52-30ec-4bd5-b875-8e57afd3a44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:12.152547 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4539f52-30ec-4bd5-b875-8e57afd3a44d admin admin] PATH_INFO: `/v3/users/d4ac39859e8c490182c99a468dd92011` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:12.153694 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4539f52-30ec-4bd5-b875-8e57afd3a44d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4ac39859e8c490182c99a468dd92011)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:12.155584 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4539f52-30ec-4bd5-b875-8e57afd3a44d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:12.180349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ce736a31-78a6-4760-a8a1-5a10ac2e7b86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e510e9cdaa4457982ce1bc7af9aeee1', 'request_id': 'req-ce736a31-78a6-4760-a8a1-5a10ac2e7b86'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:12.187112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ce736a31-78a6-4760-a8a1-5a10ac2e7b86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e510e9cdaa4457982ce1bc7af9aeee1', 'request_id': 'req-ce736a31-78a6-4760-a8a1-5a10ac2e7b86'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:12.192731 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ce736a31-78a6-4760-a8a1-5a10ac2e7b86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e510e9cdaa4457982ce1bc7af9aeee1', 'request_id': 'req-ce736a31-78a6-4760-a8a1-5a10ac2e7b86'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:12.276668 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ce736a31-78a6-4760-a8a1-5a10ac2e7b86 admin admin] The token cache is being invalidate because project 4e510e9cdaa4457982ce1bc7af9aeee1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:49:12.276960 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ce736a31-78a6-4760-a8a1-5a10ac2e7b86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:12.277630 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ce736a31-78a6-4760-a8a1-5a10ac2e7b86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:12.279156 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1308/2632] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:49:11 2022] DELETE /identity/v3/projects/4e510e9cdaa4457982ce1bc7af9aeee1 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:12.284678 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1675c18-8bb0-4fc3-88c3-541bc044ae50 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:12.284961 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1675c18-8bb0-4fc3-88c3-541bc044ae50 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:12.285263 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1675c18-8bb0-4fc3-88c3-541bc044ae50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:12.312785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-c1675c18-8bb0-4fc3-88c3-541bc044ae50 None None] MFA Rules not processed for user `587a257431cc44e098607ac45eead615`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:12.314009 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c1675c18-8bb0-4fc3-88c3-541bc044ae50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:12.377454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1309/2633] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:12.378680 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74aebe-c1cc-415b-9a5f-32a0e86a74f1 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:12.378965 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74aebe-c1cc-415b-9a5f-32a0e86a74f1 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:12.379241 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74aebe-c1cc-415b-9a5f-32a0e86a74f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:12.400082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-0f74aebe-c1cc-415b-9a5f-32a0e86a74f1 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:12.401414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0f74aebe-c1cc-415b-9a5f-32a0e86a74f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:12.411269 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d4539f52-30ec-4bd5-b875-8e57afd3a44d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4ac39859e8c490182c99a468dd92011'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:12.412042 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d4539f52-30ec-4bd5-b875-8e57afd3a44d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4ac39859e8c490182c99a468dd92011'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:12.412931 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d4539f52-30ec-4bd5-b875-8e57afd3a44d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4ac39859e8c490182c99a468dd92011'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:12.420150 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d4539f52-30ec-4bd5-b875-8e57afd3a44d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4ac39859e8c490182c99a468dd92011'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:12.467472 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1310/2634] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:12.468512 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74aebe-c1cc-415b-9a5f-32a0e86a74f1 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:12.469948 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0f74aebe-c1cc-415b-9a5f-32a0e86a74f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:12.525916 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74aebe-c1cc-415b-9a5f-32a0e86a74f1 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:12.529465 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1502d91-be0f-4f00-b8cf-81d63c47e492 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:12.530059 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1502d91-be0f-4f00-b8cf-81d63c47e492 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:12.530333 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1502d91-be0f-4f00-b8cf-81d63c47e492 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:12.530626 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1502d91-be0f-4f00-b8cf-81d63c47e492 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:12.532317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b1502d91-be0f-4f00-b8cf-81d63c47e492 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:12.542827 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1502d91-be0f-4f00-b8cf-81d63c47e492 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:12.544248 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1502d91-be0f-4f00-b8cf-81d63c47e492 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:12.544753 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-d4539f52-30ec-4bd5-b875-8e57afd3a44d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4ac39859e8c490182c99a468dd92011'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:12.553304 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1311/2635] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:49:12.554775 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1b5d5-59b4-4d94-a159-0a6ddb77dee6 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:12.555328 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1b5d5-59b4-4d94-a159-0a6ddb77dee6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:12.555673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1b5d5-59b4-4d94-a159-0a6ddb77dee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:12.568479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:12.573530 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1325/2636] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:49:12 2022] DELETE /identity/v3/users/d4ac39859e8c490182c99a468dd92011 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:12.577244 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-d0e1b5d5-59b4-4d94-a159-0a6ddb77dee6 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:12.578136 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a4b49-7ffd-42c0-bfb3-5e95a34fb5b4 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:12.578434 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d0e1b5d5-59b4-4d94-a159-0a6ddb77dee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:12.578607 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a4b49-7ffd-42c0-bfb3-5e95a34fb5b4 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:12.578978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a4b49-7ffd-42c0-bfb3-5e95a34fb5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:12.586917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:12.599396 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-521a4b49-7ffd-42c0-bfb3-5e95a34fb5b4 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:12.600636 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-521a4b49-7ffd-42c0-bfb3-5e95a34fb5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:12.638502 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1312/2637] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:12.642141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e1b5d5-59b4-4d94-a159-0a6ddb77dee6 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:12.642141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d0e1b5d5-59b4-4d94-a159-0a6ddb77dee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:12.652311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e1b5d5-59b4-4d94-a159-0a6ddb77dee6 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:12.657624 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae3e5e-4116-4d6d-ad7b-a7440e548844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:12.658561 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae3e5e-4116-4d6d-ad7b-a7440e548844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:12.659044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae3e5e-4116-4d6d-ad7b-a7440e548844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:12.659512 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae3e5e-4116-4d6d-ad7b-a7440e548844 admin admin] PATH_INFO: `/v3/users/587a257431cc44e098607ac45eead615` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:12.659801 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1326/2638] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:49:12.661713 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ae3e5e-4116-4d6d-ad7b-a7440e548844 admin admin] RBAC: Authorizing `identity:delete_user(user_id=587a257431cc44e098607ac45eead615)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:12.664361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a4b49-7ffd-42c0-bfb3-5e95a34fb5b4 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:12.666869 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ae3e5e-4116-4d6d-ad7b-a7440e548844 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:12.674028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a4b49-7ffd-42c0-bfb3-5e95a34fb5b4 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:12.679581 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d1829c-8223-4d1b-8052-fb9b31dc8d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:12.680287 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1829c-8223-4d1b-8052-fb9b31dc8d01 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:12.680635 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1829c-8223-4d1b-8052-fb9b31dc8d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:12.680969 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1829c-8223-4d1b-8052-fb9b31dc8d01 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:12.681724 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d1829c-8223-4d1b-8052-fb9b31dc8d01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:49:12.682092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d1829c-8223-4d1b-8052-fb9b31dc8d01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:12.683763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d1829c-8223-4d1b-8052-fb9b31dc8d01 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:12.695317 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1327/2639] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:49:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:49:12.696124 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d1829c-8223-4d1b-8052-fb9b31dc8d01 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:12.697763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-46d1829c-8223-4d1b-8052-fb9b31dc8d01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:12.753849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d1829c-8223-4d1b-8052-fb9b31dc8d01 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:12.757584 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636eaf76-19f7-4a6d-bd8e-c29d76de44d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:12.758180 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636eaf76-19f7-4a6d-bd8e-c29d76de44d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:12.758454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636eaf76-19f7-4a6d-bd8e-c29d76de44d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:12.758719 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636eaf76-19f7-4a6d-bd8e-c29d76de44d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:12.767792 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636eaf76-19f7-4a6d-bd8e-c29d76de44d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:12.771694 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636eaf76-19f7-4a6d-bd8e-c29d76de44d0 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:12.779941 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1328/2640] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:49:12.780645 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636eaf76-19f7-4a6d-bd8e-c29d76de44d0 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:12.790017 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636eaf76-19f7-4a6d-bd8e-c29d76de44d0 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:12.794714 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca477f77-53ec-4734-aa05-b0a7306caf3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:12.795183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca477f77-53ec-4734-aa05-b0a7306caf3f admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:12.795489 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca477f77-53ec-4734-aa05-b0a7306caf3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:12.795724 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca477f77-53ec-4734-aa05-b0a7306caf3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:12.796330 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca477f77-53ec-4734-aa05-b0a7306caf3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:12.798242 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca477f77-53ec-4734-aa05-b0a7306caf3f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:12.864202 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-57ae3e5e-4116-4d6d-ad7b-a7440e548844 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '587a257431cc44e098607ac45eead615'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:12.864882 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-57ae3e5e-4116-4d6d-ad7b-a7440e548844 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '587a257431cc44e098607ac45eead615'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:12.865608 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-57ae3e5e-4116-4d6d-ad7b-a7440e548844 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '587a257431cc44e098607ac45eead615'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:12.874160 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-57ae3e5e-4116-4d6d-ad7b-a7440e548844 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '587a257431cc44e098607ac45eead615'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:12.901132 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1329/2641] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:49:12 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:49:12.905447 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca477f77-53ec-4734-aa05-b0a7306caf3f admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:12.907293 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ca477f77-53ec-4734-aa05-b0a7306caf3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:12.951311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-57ae3e5e-4116-4d6d-ad7b-a7440e548844 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '587a257431cc44e098607ac45eead615'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:12.976569 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1313/2642] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:49:12 2022] DELETE /identity/v3/users/587a257431cc44e098607ac45eead615 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:12.978934 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae3e5e-4116-4d6d-ad7b-a7440e548844 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:12.980327 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-57ae3e5e-4116-4d6d-ad7b-a7440e548844 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:12.987915 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca477f77-53ec-4734-aa05-b0a7306caf3f admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:13.054907 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae3e5e-4116-4d6d-ad7b-a7440e548844 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:13.060802 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dfa606-36e1-403f-a607-cdad0f5ae0b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:13.061461 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dfa606-36e1-403f-a607-cdad0f5ae0b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:13.061743 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dfa606-36e1-403f-a607-cdad0f5ae0b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:13.062146 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dfa606-36e1-403f-a607-cdad0f5ae0b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:13.064367 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e5dfa606-36e1-403f-a607-cdad0f5ae0b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:13.071271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72e7387-0f1d-4e37-842c-e9cf3aa1e220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:13.072140 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e7387-0f1d-4e37-842c-e9cf3aa1e220 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:13.072814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e7387-0f1d-4e37-842c-e9cf3aa1e220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:13.072814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e7387-0f1d-4e37-842c-e9cf3aa1e220 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:13.073637 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72e7387-0f1d-4e37-842c-e9cf3aa1e220 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:13.076223 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72e7387-0f1d-4e37-842c-e9cf3aa1e220 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:13.083269 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:13.106410 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:13.141871 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dfa606-36e1-403f-a607-cdad0f5ae0b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:13.143356 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dfa606-36e1-403f-a607-cdad0f5ae0b7 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:13.153588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1314/2643] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:12 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:49:13.154868 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c41f7-10ff-4e3f-bc79-8c5736e43fec None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:13.155162 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c41f7-10ff-4e3f-bc79-8c5736e43fec None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:13.155441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c41f7-10ff-4e3f-bc79-8c5736e43fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:13.157621 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1330/2644] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:49:12 2022] POST /identity/v3/users => generated 288 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:49:13.161733 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72e7387-0f1d-4e37-842c-e9cf3aa1e220 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:13.173046 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72e7387-0f1d-4e37-842c-e9cf3aa1e220 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:13.175952 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-e09c41f7-10ff-4e3f-bc79-8c5736e43fec None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:13.177172 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e09c41f7-10ff-4e3f-bc79-8c5736e43fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:13.179763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4acba-91f9-485a-8795-ba2c2696482e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:13.180518 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4acba-91f9-485a-8795-ba2c2696482e admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:13.181086 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4acba-91f9-485a-8795-ba2c2696482e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:13.188231 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4acba-91f9-485a-8795-ba2c2696482e admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:13.188231 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f4acba-91f9-485a-8795-ba2c2696482e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:13.188231 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f4acba-91f9-485a-8795-ba2c2696482e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:13.188661 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1315/2645] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:13.197860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1331/2646] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:49:13.198033 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c41f7-10ff-4e3f-bc79-8c5736e43fec None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:13.198345 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e09c41f7-10ff-4e3f-bc79-8c5736e43fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:13.198853 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4acba-91f9-485a-8795-ba2c2696482e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:13.209039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4acba-91f9-485a-8795-ba2c2696482e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:13.216068 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03760e73-c364-49fb-8edc-fb8ca2ba0fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:13.216827 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03760e73-c364-49fb-8edc-fb8ca2ba0fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:13.217177 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03760e73-c364-49fb-8edc-fb8ca2ba0fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:13.217520 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03760e73-c364-49fb-8edc-fb8ca2ba0fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:13.218271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03760e73-c364-49fb-8edc-fb8ca2ba0fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:13.220085 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03760e73-c364-49fb-8edc-fb8ca2ba0fc8 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:13.233839 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1332/2647] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:49:13.234607 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03760e73-c364-49fb-8edc-fb8ca2ba0fc8 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:13.237745 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-03760e73-c364-49fb-8edc-fb8ca2ba0fc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:13.254452 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03760e73-c364-49fb-8edc-fb8ca2ba0fc8 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:13.259779 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc297ecb-b0c2-4aa1-a0a1-cdcace98e3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:13.260523 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc297ecb-b0c2-4aa1-a0a1-cdcace98e3ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:13.260901 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc297ecb-b0c2-4aa1-a0a1-cdcace98e3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:13.261204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc297ecb-b0c2-4aa1-a0a1-cdcace98e3ec admin admin] PATH_INFO: `/v3/projects/7906051e49054eab940ef557e6da267e` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:13.262355 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc297ecb-b0c2-4aa1-a0a1-cdcace98e3ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=7906051e49054eab940ef557e6da267e)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:13.264269 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc297ecb-b0c2-4aa1-a0a1-cdcace98e3ec admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:13.269698 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c41f7-10ff-4e3f-bc79-8c5736e43fec None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:13.273553 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5273a-1f21-468f-be7c-e902a17e3eeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:13.274159 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5273a-1f21-468f-be7c-e902a17e3eeb service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:13.274441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5273a-1f21-468f-be7c-e902a17e3eeb service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:13.274713 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5273a-1f21-468f-be7c-e902a17e3eeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:13.284189 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f5273a-1f21-468f-be7c-e902a17e3eeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:13.285712 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f5273a-1f21-468f-be7c-e902a17e3eeb service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:13.297240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1316/2648] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:49:13.297240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5273a-1f21-468f-be7c-e902a17e3eeb service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:13.306448 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5273a-1f21-468f-be7c-e902a17e3eeb service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:13.311115 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb532a7-cb40-4485-8c5d-39ea7f37498b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:13.311716 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb532a7-cb40-4485-8c5d-39ea7f37498b admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:13.311993 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb532a7-cb40-4485-8c5d-39ea7f37498b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:13.312270 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb532a7-cb40-4485-8c5d-39ea7f37498b admin admin] PATH_INFO: `/v3/projects/718f988579854efe8ff0609ff2682e05/users/137b0282018e4268a2c245e945155235/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:13.328316 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb532a7-cb40-4485-8c5d-39ea7f37498b admin admin] RBAC: Authorizing `identity:create_grant(project_id=718f988579854efe8ff0609ff2682e05, user_id=137b0282018e4268a2c245e945155235, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:13.331020 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb532a7-cb40-4485-8c5d-39ea7f37498b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:13.431297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1317/2649] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:49:13 2022] PUT /identity/v3/projects/718f988579854efe8ff0609ff2682e05/users/137b0282018e4268a2c245e945155235/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:13.439106 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fc297ecb-b0c2-4aa1-a0a1-cdcace98e3ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7906051e49054eab940ef557e6da267e', 'request_id': 'req-fc297ecb-b0c2-4aa1-a0a1-cdcace98e3ec'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:13.441965 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8e8ca-7237-4685-a37f-e7a998647cfd None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:13.442255 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8e8ca-7237-4685-a37f-e7a998647cfd None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:13.442528 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8e8ca-7237-4685-a37f-e7a998647cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:13.470454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-5ae8e8ca-7237-4685-a37f-e7a998647cfd None None] MFA Rules not processed for user `137b0282018e4268a2c245e945155235`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:13.471650 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5ae8e8ca-7237-4685-a37f-e7a998647cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:13.506226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fc297ecb-b0c2-4aa1-a0a1-cdcace98e3ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7906051e49054eab940ef557e6da267e', 'request_id': 'req-fc297ecb-b0c2-4aa1-a0a1-cdcace98e3ec'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:13.515040 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fc297ecb-b0c2-4aa1-a0a1-cdcace98e3ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7906051e49054eab940ef557e6da267e', 'request_id': 'req-fc297ecb-b0c2-4aa1-a0a1-cdcace98e3ec'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:13.517193 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:13.526864 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fc297ecb-b0c2-4aa1-a0a1-cdcace98e3ec admin admin] The token cache is being invalidate because project 7906051e49054eab940ef557e6da267e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:49:13.527102 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fc297ecb-b0c2-4aa1-a0a1-cdcace98e3ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:13.527740 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fc297ecb-b0c2-4aa1-a0a1-cdcace98e3ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:13.529588 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1333/2650] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:49:13 2022] DELETE /identity/v3/projects/7906051e49054eab940ef557e6da267e => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:13.533132 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc297ecb-b0c2-4aa1-a0a1-cdcace98e3ec admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:13.534626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fc297ecb-b0c2-4aa1-a0a1-cdcace98e3ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:13.549130 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1318/2651] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:13.553563 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae8e8ca-7237-4685-a37f-e7a998647cfd None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:13.555260 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5ae8e8ca-7237-4685-a37f-e7a998647cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:13.621020 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc297ecb-b0c2-4aa1-a0a1-cdcace98e3ec admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:13.626667 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d17e2-0233-4ecf-9504-56a761488adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:13.627255 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae8e8ca-7237-4685-a37f-e7a998647cfd None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:13.627422 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d17e2-0233-4ecf-9504-56a761488adf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:13.627626 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d17e2-0233-4ecf-9504-56a761488adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:13.627904 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d17e2-0233-4ecf-9504-56a761488adf admin admin] PATH_INFO: `/v3/users/3f7c1442bb8b457da6be54d2f1dc1eb6` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:13.629033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d17e2-0233-4ecf-9504-56a761488adf admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f7c1442bb8b457da6be54d2f1dc1eb6)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:13.631227 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d17e2-0233-4ecf-9504-56a761488adf admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:13.632194 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a782a0b-04cd-4284-89f7-a2b90dbfc47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:13.632801 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a782a0b-04cd-4284-89f7-a2b90dbfc47f admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:13.633077 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a782a0b-04cd-4284-89f7-a2b90dbfc47f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:13.633463 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a782a0b-04cd-4284-89f7-a2b90dbfc47f admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:13.634084 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a782a0b-04cd-4284-89f7-a2b90dbfc47f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:13.636027 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a782a0b-04cd-4284-89f7-a2b90dbfc47f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:13.637607 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:13.827632 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1319/2652] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:49:13 2022] POST /identity/v3/projects => generated 375 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:49:13.828226 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a782a0b-04cd-4284-89f7-a2b90dbfc47f admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:13.831803 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2a782a0b-04cd-4284-89f7-a2b90dbfc47f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:13.895754 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a782a0b-04cd-4284-89f7-a2b90dbfc47f admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:13.901308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6476513-5fab-4f85-aa98-14dd57c43ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:13.901942 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6476513-5fab-4f85-aa98-14dd57c43ab3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:13.902219 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6476513-5fab-4f85-aa98-14dd57c43ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:13.902494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6476513-5fab-4f85-aa98-14dd57c43ab3 admin admin] PATH_INFO: `/v3/projects/39502bc219be4bb1bbeb04b0fcde1ca2` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:13.903688 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6476513-5fab-4f85-aa98-14dd57c43ab3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39502bc219be4bb1bbeb04b0fcde1ca2)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:13.905640 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6476513-5fab-4f85-aa98-14dd57c43ab3 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:13.916368 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:13.919042 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-643d17e2-0233-4ecf-9504-56a761488adf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f7c1442bb8b457da6be54d2f1dc1eb6'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:13.919742 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-643d17e2-0233-4ecf-9504-56a761488adf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f7c1442bb8b457da6be54d2f1dc1eb6'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:13.920367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-643d17e2-0233-4ecf-9504-56a761488adf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f7c1442bb8b457da6be54d2f1dc1eb6'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:13.928601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-643d17e2-0233-4ecf-9504-56a761488adf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f7c1442bb8b457da6be54d2f1dc1eb6'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:14.035953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d6476513-5fab-4f85-aa98-14dd57c43ab3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39502bc219be4bb1bbeb04b0fcde1ca2', 'request_id': 'req-d6476513-5fab-4f85-aa98-14dd57c43ab3'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:14.042128 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d6476513-5fab-4f85-aa98-14dd57c43ab3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39502bc219be4bb1bbeb04b0fcde1ca2', 'request_id': 'req-d6476513-5fab-4f85-aa98-14dd57c43ab3'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:14.047441 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d6476513-5fab-4f85-aa98-14dd57c43ab3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39502bc219be4bb1bbeb04b0fcde1ca2', 'request_id': 'req-d6476513-5fab-4f85-aa98-14dd57c43ab3'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:14.054230 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-643d17e2-0233-4ecf-9504-56a761488adf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f7c1442bb8b457da6be54d2f1dc1eb6'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:14.072851 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1334/2653] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:49:13 2022] DELETE /identity/v3/users/3f7c1442bb8b457da6be54d2f1dc1eb6 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:49:14.076027 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d17e2-0233-4ecf-9504-56a761488adf admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:14.076027 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-643d17e2-0233-4ecf-9504-56a761488adf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:14.144000 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d17e2-0233-4ecf-9504-56a761488adf admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:14.145432 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d6476513-5fab-4f85-aa98-14dd57c43ab3 admin admin] The token cache is being invalidate because project 39502bc219be4bb1bbeb04b0fcde1ca2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:49:14.145725 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d6476513-5fab-4f85-aa98-14dd57c43ab3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:14.146491 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-d6476513-5fab-4f85-aa98-14dd57c43ab3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:14.148821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1320/2654] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:49:13 2022] DELETE /identity/v3/projects/39502bc219be4bb1bbeb04b0fcde1ca2 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:14.149848 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4b9d6-e8de-4876-9061-bea67d514f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:14.150494 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4b9d6-e8de-4876-9061-bea67d514f14 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:14.150771 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4b9d6-e8de-4876-9061-bea67d514f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:14.151037 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4b9d6-e8de-4876-9061-bea67d514f14 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:14.151742 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a4b9d6-e8de-4876-9061-bea67d514f14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:14.153607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73ada9-e98a-46b7-a51d-1b406d623a34 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:14.154090 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a4b9d6-e8de-4876-9061-bea67d514f14 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:14.154394 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73ada9-e98a-46b7-a51d-1b406d623a34 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:14.155572 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73ada9-e98a-46b7-a51d-1b406d623a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:14.177599 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-1e73ada9-e98a-46b7-a51d-1b406d623a34 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:14.179444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1e73ada9-e98a-46b7-a51d-1b406d623a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:14.187593 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1321/2655] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:14.195129 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6febb-e827-46f3-8d00-b034818f9af6 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:14.195406 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6febb-e827-46f3-8d00-b034818f9af6 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:14.195674 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6febb-e827-46f3-8d00-b034818f9af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:14.215024 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-3bb6febb-e827-46f3-8d00-b034818f9af6 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:14.216231 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-3bb6febb-e827-46f3-8d00-b034818f9af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:14.223994 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1322/2656] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:49:14.230473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6febb-e827-46f3-8d00-b034818f9af6 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:14.239230 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6febb-e827-46f3-8d00-b034818f9af6 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:14.244032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ab78f-ba85-4280-88c9-78bf9ea99129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:14.244595 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ab78f-ba85-4280-88c9-78bf9ea99129 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:14.244870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ab78f-ba85-4280-88c9-78bf9ea99129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:14.245156 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ab78f-ba85-4280-88c9-78bf9ea99129 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:14.245848 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ab78f-ba85-4280-88c9-78bf9ea99129 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:49:14.246224 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ab78f-ba85-4280-88c9-78bf9ea99129 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:14.250555 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ab78f-ba85-4280-88c9-78bf9ea99129 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:14.258980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1323/2657] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:49:14.262920 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ab78f-ba85-4280-88c9-78bf9ea99129 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:14.272544 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ab78f-ba85-4280-88c9-78bf9ea99129 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:14.277258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a966731-9554-4d6d-a14e-ce290b371ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:14.277807 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a966731-9554-4d6d-a14e-ce290b371ae7 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:14.278084 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a966731-9554-4d6d-a14e-ce290b371ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:14.278352 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a966731-9554-4d6d-a14e-ce290b371ae7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:14.278938 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a966731-9554-4d6d-a14e-ce290b371ae7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:14.280797 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a966731-9554-4d6d-a14e-ce290b371ae7 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:14.301184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1335/2658] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:49:14 2022] POST /identity/v3/users => generated 294 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:49:14.304622 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4b9d6-e8de-4876-9061-bea67d514f14 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:14.305844 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-81a4b9d6-e8de-4876-9061-bea67d514f14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:14.318155 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4b9d6-e8de-4876-9061-bea67d514f14 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:14.324072 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556058b5-b5b9-489f-a937-cfea9b1e779c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:14.324790 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556058b5-b5b9-489f-a937-cfea9b1e779c admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:14.325130 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556058b5-b5b9-489f-a937-cfea9b1e779c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:14.325445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556058b5-b5b9-489f-a937-cfea9b1e779c admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:14.326084 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556058b5-b5b9-489f-a937-cfea9b1e779c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:14.327713 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556058b5-b5b9-489f-a937-cfea9b1e779c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:14.338088 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1336/2659] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:49:14.349903 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556058b5-b5b9-489f-a937-cfea9b1e779c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:14.360588 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556058b5-b5b9-489f-a937-cfea9b1e779c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:14.366820 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83d5a0-daba-4f5b-84d8-f4998f3765ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:14.367840 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83d5a0-daba-4f5b-84d8-f4998f3765ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:14.368383 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83d5a0-daba-4f5b-84d8-f4998f3765ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:14.368666 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83d5a0-daba-4f5b-84d8-f4998f3765ed admin admin] PATH_INFO: `/v3/projects/ba3bb0c70dcc440287cf93d962ea6795/users/d5ee1b1414fa468cb55bd1d9143a8e28/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:14.373775 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1324/2660] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:49:14 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:49:14.374833 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a966731-9554-4d6d-a14e-ce290b371ae7 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:14.388336 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be83d5a0-daba-4f5b-84d8-f4998f3765ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba3bb0c70dcc440287cf93d962ea6795, user_id=d5ee1b1414fa468cb55bd1d9143a8e28, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:14.391335 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be83d5a0-daba-4f5b-84d8-f4998f3765ed admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:14.450362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a966731-9554-4d6d-a14e-ce290b371ae7 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:14.456117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd4d9b-47ce-4dfa-9ac8-9f2e428008a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:14.456763 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd4d9b-47ce-4dfa-9ac8-9f2e428008a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:14.457196 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd4d9b-47ce-4dfa-9ac8-9f2e428008a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:14.457477 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd4d9b-47ce-4dfa-9ac8-9f2e428008a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:14.458187 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebd4d9b-47ce-4dfa-9ac8-9f2e428008a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:14.460285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebd4d9b-47ce-4dfa-9ac8-9f2e428008a6 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:14.466725 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1337/2661] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:49:14 2022] PUT /identity/v3/projects/ba3bb0c70dcc440287cf93d962ea6795/users/d5ee1b1414fa468cb55bd1d9143a8e28/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:49:14.471899 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83d5a0-daba-4f5b-84d8-f4998f3765ed admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:14.519002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:14.539687 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83d5a0-daba-4f5b-84d8-f4998f3765ed admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:14.544832 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd847de-56e8-4aeb-82b3-2bca0f13e67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:14.545559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd847de-56e8-4aeb-82b3-2bca0f13e67d admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:14.545952 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd847de-56e8-4aeb-82b3-2bca0f13e67d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:14.546332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd847de-56e8-4aeb-82b3-2bca0f13e67d admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:14.547335 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1325/2662] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:49:14 2022] POST /identity/v3/users => generated 292 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:49:14.550062 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd847de-56e8-4aeb-82b3-2bca0f13e67d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:14.551131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd4d9b-47ce-4dfa-9ac8-9f2e428008a6 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:14.551748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd847de-56e8-4aeb-82b3-2bca0f13e67d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:14.562081 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd4d9b-47ce-4dfa-9ac8-9f2e428008a6 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:14.565328 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1338/2663] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:49:14.569703 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd847de-56e8-4aeb-82b3-2bca0f13e67d admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:14.571911 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b277f2-6fd6-4558-8c13-71dd0ba547da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:14.572744 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b277f2-6fd6-4558-8c13-71dd0ba547da admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:14.573021 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b277f2-6fd6-4558-8c13-71dd0ba547da admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:14.573764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b277f2-6fd6-4558-8c13-71dd0ba547da admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:14.574423 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b277f2-6fd6-4558-8c13-71dd0ba547da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:14.576011 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b277f2-6fd6-4558-8c13-71dd0ba547da admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:14.582566 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd847de-56e8-4aeb-82b3-2bca0f13e67d admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:14.587708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7f0df-d59e-43e2-a5c3-1daf6cb15fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:14.588374 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7f0df-d59e-43e2-a5c3-1daf6cb15fe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:14.588651 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7f0df-d59e-43e2-a5c3-1daf6cb15fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:14.591945 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7f0df-d59e-43e2-a5c3-1daf6cb15fe5 admin admin] PATH_INFO: `/v3/projects/ba3bb0c70dcc440287cf93d962ea6795/users/d5ee1b1414fa468cb55bd1d9143a8e28/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:14.592087 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1326/2664] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:49:14.592087 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b277f2-6fd6-4558-8c13-71dd0ba547da admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:14.592300 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7f0df-d59e-43e2-a5c3-1daf6cb15fe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba3bb0c70dcc440287cf93d962ea6795, user_id=d5ee1b1414fa468cb55bd1d9143a8e28, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:14.592454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-60b277f2-6fd6-4558-8c13-71dd0ba547da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:14.594773 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7f0df-d59e-43e2-a5c3-1daf6cb15fe5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:14.608116 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b277f2-6fd6-4558-8c13-71dd0ba547da admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:14.609226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1339/2665] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:49:14 2022] PUT /identity/v3/projects/ba3bb0c70dcc440287cf93d962ea6795/users/d5ee1b1414fa468cb55bd1d9143a8e28/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:14.610372 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7f0df-d59e-43e2-a5c3-1daf6cb15fe5 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:14.635950 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:14.667750 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e10018-e698-4abf-9efd-44d667506c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:14.668395 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e10018-e698-4abf-9efd-44d667506c13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:14.668759 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e10018-e698-4abf-9efd-44d667506c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:14.669040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e10018-e698-4abf-9efd-44d667506c13 admin admin] PATH_INFO: `/v3/projects/7b1f2f9c73a34fae9c538aa64ef014c3` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:14.670482 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e10018-e698-4abf-9efd-44d667506c13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b1f2f9c73a34fae9c538aa64ef014c3)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:14.672386 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e10018-e698-4abf-9efd-44d667506c13 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:14.680269 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7f0df-d59e-43e2-a5c3-1daf6cb15fe5 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:14.685165 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63acad04-0a84-43b7-80a5-8cb2efe00c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:14.685802 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63acad04-0a84-43b7-80a5-8cb2efe00c71 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:14.686079 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63acad04-0a84-43b7-80a5-8cb2efe00c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:14.686390 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63acad04-0a84-43b7-80a5-8cb2efe00c71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:14.687033 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63acad04-0a84-43b7-80a5-8cb2efe00c71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:14.688638 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63acad04-0a84-43b7-80a5-8cb2efe00c71 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:14.702150 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1340/2666] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:49:14.702150 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08eb8b-4d47-4b6a-98eb-14c8b2cb03fe None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:14.702150 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08eb8b-4d47-4b6a-98eb-14c8b2cb03fe None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:14.702150 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08eb8b-4d47-4b6a-98eb-14c8b2cb03fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:14.728999 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-4e08eb8b-4d47-4b6a-98eb-14c8b2cb03fe None None] MFA Rules not processed for user `d5ee1b1414fa468cb55bd1d9143a8e28`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:14.730228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4e08eb8b-4d47-4b6a-98eb-14c8b2cb03fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:14.799573 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-45e10018-e698-4abf-9efd-44d667506c13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b1f2f9c73a34fae9c538aa64ef014c3', 'request_id': 'req-45e10018-e698-4abf-9efd-44d667506c13'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:14.799820 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1341/2667] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:14.800639 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08eb8b-4d47-4b6a-98eb-14c8b2cb03fe None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:14.810041 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-45e10018-e698-4abf-9efd-44d667506c13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b1f2f9c73a34fae9c538aa64ef014c3', 'request_id': 'req-45e10018-e698-4abf-9efd-44d667506c13'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:14.811554 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08eb8b-4d47-4b6a-98eb-14c8b2cb03fe None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:14.816310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-45e10018-e698-4abf-9efd-44d667506c13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b1f2f9c73a34fae9c538aa64ef014c3', 'request_id': 'req-45e10018-e698-4abf-9efd-44d667506c13'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:14.868894 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395cdd50-6819-4600-93d2-b17d4c519a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:14.869583 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395cdd50-6819-4600-93d2-b17d4c519a13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:14.869861 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395cdd50-6819-4600-93d2-b17d4c519a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:14.870130 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395cdd50-6819-4600-93d2-b17d4c519a13 admin admin] PATH_INFO: `/v3/projects/b0ea80c4a39f47e1a9f3d050c46fe2bb/users/d89b729776114f7ab97c7a99f5a777c2/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:14.885459 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395cdd50-6819-4600-93d2-b17d4c519a13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0ea80c4a39f47e1a9f3d050c46fe2bb, user_id=d89b729776114f7ab97c7a99f5a777c2, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:14.888167 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395cdd50-6819-4600-93d2-b17d4c519a13 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:14.900174 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-45e10018-e698-4abf-9efd-44d667506c13 admin admin] The token cache is being invalidate because project 7b1f2f9c73a34fae9c538aa64ef014c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:49:14.910386 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-45e10018-e698-4abf-9efd-44d667506c13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:14.910386 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-45e10018-e698-4abf-9efd-44d667506c13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:14.910386 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1327/2668] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:49:14 2022] DELETE /identity/v3/projects/7b1f2f9c73a34fae9c538aa64ef014c3 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:14.910386 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e10018-e698-4abf-9efd-44d667506c13 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:14.910386 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-45e10018-e698-4abf-9efd-44d667506c13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:14.962051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e10018-e698-4abf-9efd-44d667506c13 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:14.967200 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1342/2669] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:49:14 2022] PUT /identity/v3/projects/b0ea80c4a39f47e1a9f3d050c46fe2bb/users/d89b729776114f7ab97c7a99f5a777c2/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:14.968522 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd478c-fff7-4d5a-b2c0-e663eed6ef03 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:14.968808 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd478c-fff7-4d5a-b2c0-e663eed6ef03 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:14.969095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd478c-fff7-4d5a-b2c0-e663eed6ef03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:14.988458 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-32fd478c-fff7-4d5a-b2c0-e663eed6ef03 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:14.989733 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-32fd478c-fff7-4d5a-b2c0-e663eed6ef03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:15.018114 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed033f2-07b2-4db8-b8c8-87818fb5136f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:15.018723 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed033f2-07b2-4db8-b8c8-87818fb5136f service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:15.019050 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed033f2-07b2-4db8-b8c8-87818fb5136f service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:15.019369 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed033f2-07b2-4db8-b8c8-87818fb5136f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:15.021322 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0ed033f2-07b2-4db8-b8c8-87818fb5136f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:15.066394 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1343/2670] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:15.066394 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc5f06-3199-4d0d-ae05-ace0590d4f97 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:15.066394 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc5f06-3199-4d0d-ae05-ace0590d4f97 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:15.066394 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc5f06-3199-4d0d-ae05-ace0590d4f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:15.093078 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d2bc5f06-3199-4d0d-ae05-ace0590d4f97 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:15.095357 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d2bc5f06-3199-4d0d-ae05-ace0590d4f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:15.095807 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed033f2-07b2-4db8-b8c8-87818fb5136f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:15.097221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed033f2-07b2-4db8-b8c8-87818fb5136f service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:15.106862 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1328/2671] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:49:15.112555 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189c077-91ed-4abb-b64a-fad16bc2f31a None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:15.112555 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189c077-91ed-4abb-b64a-fad16bc2f31a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:15.112555 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189c077-91ed-4abb-b64a-fad16bc2f31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:15.113020 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1344/2672] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:15.125416 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc5f06-3199-4d0d-ae05-ace0590d4f97 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:15.127890 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d2bc5f06-3199-4d0d-ae05-ace0590d4f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:15.138144 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-6189c077-91ed-4abb-b64a-fad16bc2f31a None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:15.139554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6189c077-91ed-4abb-b64a-fad16bc2f31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:15.157579 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1329/2673] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:15.161976 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189c077-91ed-4abb-b64a-fad16bc2f31a None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:15.173160 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189c077-91ed-4abb-b64a-fad16bc2f31a None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:15.182005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01532183-4be9-48ee-8006-3811a13a81f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:15.182850 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01532183-4be9-48ee-8006-3811a13a81f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:15.183112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01532183-4be9-48ee-8006-3811a13a81f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:15.183578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01532183-4be9-48ee-8006-3811a13a81f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:15.184250 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01532183-4be9-48ee-8006-3811a13a81f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:49:15.185180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01532183-4be9-48ee-8006-3811a13a81f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:15.187295 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01532183-4be9-48ee-8006-3811a13a81f6 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:15.187948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:15.201183 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1330/2674] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:49:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:49:15.206310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01532183-4be9-48ee-8006-3811a13a81f6 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:15.208676 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc5f06-3199-4d0d-ae05-ace0590d4f97 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:15.214005 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1efbc1-1eb2-45dc-9896-5043443e7e0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:15.214906 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1efbc1-1eb2-45dc-9896-5043443e7e0c service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:15.215186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1efbc1-1eb2-45dc-9896-5043443e7e0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:15.215752 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1efbc1-1eb2-45dc-9896-5043443e7e0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:15.216794 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01532183-4be9-48ee-8006-3811a13a81f6 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:15.222665 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8b72c-9578-4a52-80c3-e5f12dae9b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:15.223262 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8b72c-9578-4a52-80c3-e5f12dae9b51 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:15.223815 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8b72c-9578-4a52-80c3-e5f12dae9b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:15.224093 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8b72c-9578-4a52-80c3-e5f12dae9b51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:15.224711 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f8b72c-9578-4a52-80c3-e5f12dae9b51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:15.226120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1efbc1-1eb2-45dc-9896-5043443e7e0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:15.227426 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f8b72c-9578-4a52-80c3-e5f12dae9b51 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:15.227785 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1efbc1-1eb2-45dc-9896-5043443e7e0c service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:15.238353 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1345/2675] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:49:15.265390 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:15.318115 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1331/2676] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:49:15 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:49:15.322806 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1efbc1-1eb2-45dc-9896-5043443e7e0c service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:15.332630 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1efbc1-1eb2-45dc-9896-5043443e7e0c service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:15.396620 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af1bac-bbcf-4bca-948f-185bfd632584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:15.397991 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af1bac-bbcf-4bca-948f-185bfd632584 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:15.398295 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af1bac-bbcf-4bca-948f-185bfd632584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:15.398566 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af1bac-bbcf-4bca-948f-185bfd632584 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:15.399195 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91af1bac-bbcf-4bca-948f-185bfd632584 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:15.401537 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91af1bac-bbcf-4bca-948f-185bfd632584 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:15.442194 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:15.494149 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1346/2677] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:49:15 2022] POST /identity/v3/users => generated 290 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:49:15.497855 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8b72c-9578-4a52-80c3-e5f12dae9b51 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:15.509265 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8b72c-9578-4a52-80c3-e5f12dae9b51 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:15.515436 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd8a352-3cfe-45a3-be40-40f71302b18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:15.516167 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8a352-3cfe-45a3-be40-40f71302b18d admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:15.520009 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8a352-3cfe-45a3-be40-40f71302b18d admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:15.520009 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8a352-3cfe-45a3-be40-40f71302b18d admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:15.520009 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd8a352-3cfe-45a3-be40-40f71302b18d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:15.520009 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd8a352-3cfe-45a3-be40-40f71302b18d admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:15.529870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1332/2678] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:49:15.534490 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af1bac-bbcf-4bca-948f-185bfd632584 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:15.544912 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af1bac-bbcf-4bca-948f-185bfd632584 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:15.550345 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850cfd94-6f64-4c1d-be7b-3760174ef548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:15.551976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cfd94-6f64-4c1d-be7b-3760174ef548 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:15.552257 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cfd94-6f64-4c1d-be7b-3760174ef548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:15.552526 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cfd94-6f64-4c1d-be7b-3760174ef548 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:15.553478 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850cfd94-6f64-4c1d-be7b-3760174ef548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:15.555500 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850cfd94-6f64-4c1d-be7b-3760174ef548 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:15.568171 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1347/2679] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:49:15.570240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd8a352-3cfe-45a3-be40-40f71302b18d admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:15.580426 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd8a352-3cfe-45a3-be40-40f71302b18d admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:15.586031 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97966293-fd80-4e67-82d7-fafcbde59421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:15.586685 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97966293-fd80-4e67-82d7-fafcbde59421 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:15.587087 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97966293-fd80-4e67-82d7-fafcbde59421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:15.587367 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97966293-fd80-4e67-82d7-fafcbde59421 admin admin] PATH_INFO: `/v3/projects/2216c5cb9f184935960b2d3880aaf25f/users/5db25d6359594956b721f6a6525cec96/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:15.602765 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97966293-fd80-4e67-82d7-fafcbde59421 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2216c5cb9f184935960b2d3880aaf25f, user_id=5db25d6359594956b721f6a6525cec96, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:15.605706 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97966293-fd80-4e67-82d7-fafcbde59421 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:15.671609 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1333/2680] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:49:15 2022] PUT /identity/v3/projects/2216c5cb9f184935960b2d3880aaf25f/users/5db25d6359594956b721f6a6525cec96/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:15.682444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d1adc-ac0f-43e1-8f06-dcd79c4b1fc5 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:15.682730 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d1adc-ac0f-43e1-8f06-dcd79c4b1fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:15.683031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d1adc-ac0f-43e1-8f06-dcd79c4b1fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:15.711689 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b53d1adc-ac0f-43e1-8f06-dcd79c4b1fc5 None None] MFA Rules not processed for user `5db25d6359594956b721f6a6525cec96`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:15.712881 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b53d1adc-ac0f-43e1-8f06-dcd79c4b1fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:15.796485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1348/2681] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:15.801039 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97966293-fd80-4e67-82d7-fafcbde59421 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:15.872915 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97966293-fd80-4e67-82d7-fafcbde59421 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:15.878638 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9041b6f-5886-44bc-9c26-7ab7f310882a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:15.879285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9041b6f-5886-44bc-9c26-7ab7f310882a admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:15.879577 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9041b6f-5886-44bc-9c26-7ab7f310882a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:15.879803 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9041b6f-5886-44bc-9c26-7ab7f310882a admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:15.880442 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9041b6f-5886-44bc-9c26-7ab7f310882a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:15.882790 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9041b6f-5886-44bc-9c26-7ab7f310882a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:15.963832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1334/2682] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:49:15 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:49:15.967856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d1adc-ac0f-43e1-8f06-dcd79c4b1fc5 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:15.978814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d1adc-ac0f-43e1-8f06-dcd79c4b1fc5 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:16.038697 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2edf0-14ed-4185-82cb-76a77b0d5d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:16.039714 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2edf0-14ed-4185-82cb-76a77b0d5d78 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:16.039916 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2edf0-14ed-4185-82cb-76a77b0d5d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:16.040258 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2edf0-14ed-4185-82cb-76a77b0d5d78 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:16.040956 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2edf0-14ed-4185-82cb-76a77b0d5d78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:16.042977 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2edf0-14ed-4185-82cb-76a77b0d5d78 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:16.144679 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1349/2683] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:49:15 2022] POST /identity/v3/users => generated 290 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:49:16.149357 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9041b6f-5886-44bc-9c26-7ab7f310882a admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:16.163788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9041b6f-5886-44bc-9c26-7ab7f310882a admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:16.168488 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f44f58-d94d-4047-b290-e568b3cc8573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:16.169170 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f44f58-d94d-4047-b290-e568b3cc8573 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:16.169473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f44f58-d94d-4047-b290-e568b3cc8573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:16.171013 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f44f58-d94d-4047-b290-e568b3cc8573 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:16.171013 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f44f58-d94d-4047-b290-e568b3cc8573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:16.171906 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f44f58-d94d-4047-b290-e568b3cc8573 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:16.183133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1335/2684] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:49:16.186851 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2edf0-14ed-4185-82cb-76a77b0d5d78 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:16.199274 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2edf0-14ed-4185-82cb-76a77b0d5d78 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:16.203915 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396cb2fb-e444-4b90-87a6-a79c9d8a274d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:16.204514 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396cb2fb-e444-4b90-87a6-a79c9d8a274d admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:16.204809 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396cb2fb-e444-4b90-87a6-a79c9d8a274d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:16.205065 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396cb2fb-e444-4b90-87a6-a79c9d8a274d admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:16.205742 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396cb2fb-e444-4b90-87a6-a79c9d8a274d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:16.207319 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396cb2fb-e444-4b90-87a6-a79c9d8a274d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:16.217435 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1350/2685] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:49:16.221708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f44f58-d94d-4047-b290-e568b3cc8573 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:16.232310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f44f58-d94d-4047-b290-e568b3cc8573 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:16.237018 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab51b6-2cbb-4072-ba37-87846fa4ded0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:16.237701 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab51b6-2cbb-4072-ba37-87846fa4ded0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:16.237983 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab51b6-2cbb-4072-ba37-87846fa4ded0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:16.238258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab51b6-2cbb-4072-ba37-87846fa4ded0 admin admin] PATH_INFO: `/v3/projects/c2ebdf5cc5e54cec9d09cde288d0d273/users/81f7903718ca47aebf41179758763e01/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:16.253889 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab51b6-2cbb-4072-ba37-87846fa4ded0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2ebdf5cc5e54cec9d09cde288d0d273, user_id=81f7903718ca47aebf41179758763e01, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:16.256565 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab51b6-2cbb-4072-ba37-87846fa4ded0 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:16.335768 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1336/2686] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:49:16 2022] PUT /identity/v3/projects/c2ebdf5cc5e54cec9d09cde288d0d273/users/81f7903718ca47aebf41179758763e01/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:16.345865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1d93c-e61e-4dd5-9672-78164ab4ea04 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:16.346231 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1d93c-e61e-4dd5-9672-78164ab4ea04 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:16.346572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1d93c-e61e-4dd5-9672-78164ab4ea04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:16.376762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-c5b1d93c-e61e-4dd5-9672-78164ab4ea04 None None] MFA Rules not processed for user `81f7903718ca47aebf41179758763e01`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:16.378062 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c5b1d93c-e61e-4dd5-9672-78164ab4ea04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:16.425771 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:16.449250 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1351/2687] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:16.453678 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab51b6-2cbb-4072-ba37-87846fa4ded0 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:16.458100 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:16.520552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab51b6-2cbb-4072-ba37-87846fa4ded0 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:16.525417 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89c2e8-f50b-4ce3-8cd8-faf68a01174b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:16.526051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89c2e8-f50b-4ce3-8cd8-faf68a01174b admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:16.526383 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89c2e8-f50b-4ce3-8cd8-faf68a01174b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:16.526654 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89c2e8-f50b-4ce3-8cd8-faf68a01174b admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:16.527292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f89c2e8-f50b-4ce3-8cd8-faf68a01174b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:16.529251 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f89c2e8-f50b-4ce3-8cd8-faf68a01174b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:16.558488 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598a248-16cf-45e1-ac58-472582321fcb None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:16.558980 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598a248-16cf-45e1-ac58-472582321fcb None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:16.559080 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598a248-16cf-45e1-ac58-472582321fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:16.591215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-1598a248-16cf-45e1-ac58-472582321fcb None None] MFA Rules not processed for user `d89b729776114f7ab97c7a99f5a777c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:16.592595 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1598a248-16cf-45e1-ac58-472582321fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:16.664215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1352/2688] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:49:16.677011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598a248-16cf-45e1-ac58-472582321fcb None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:16.678968 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1337/2689] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:49:16 2022] POST /identity/v3/projects => generated 379 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:49:16.682876 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89c2e8-f50b-4ce3-8cd8-faf68a01174b admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:16.751461 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89c2e8-f50b-4ce3-8cd8-faf68a01174b admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:16.756558 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d077225b-d5bb-4b68-bccd-ba158772ffc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:16.757379 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d077225b-d5bb-4b68-bccd-ba158772ffc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:16.757842 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d077225b-d5bb-4b68-bccd-ba158772ffc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:16.758214 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d077225b-d5bb-4b68-bccd-ba158772ffc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:16.758891 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d077225b-d5bb-4b68-bccd-ba158772ffc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:16.760851 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d077225b-d5bb-4b68-bccd-ba158772ffc4 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:16.842893 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1338/2690] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:49:16 2022] POST /identity/v3/users => generated 296 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:49:16.846833 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d077225b-d5bb-4b68-bccd-ba158772ffc4 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:16.857176 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d077225b-d5bb-4b68-bccd-ba158772ffc4 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:16.862362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec87ff9-07e4-48c3-a8a9-ab27e1834b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:16.863107 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec87ff9-07e4-48c3-a8a9-ab27e1834b15 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:16.863391 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec87ff9-07e4-48c3-a8a9-ab27e1834b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:16.863671 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec87ff9-07e4-48c3-a8a9-ab27e1834b15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:16.868792 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec87ff9-07e4-48c3-a8a9-ab27e1834b15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:16.868792 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec87ff9-07e4-48c3-a8a9-ab27e1834b15 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:16.874613 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598a248-16cf-45e1-ac58-472582321fcb None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:16.880873 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1339/2691] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:49:16.885379 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec87ff9-07e4-48c3-a8a9-ab27e1834b15 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:16.895485 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec87ff9-07e4-48c3-a8a9-ab27e1834b15 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:16.902252 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aaf359-c6e9-42dc-a9e9-f7b2a05cd971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:16.902912 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aaf359-c6e9-42dc-a9e9-f7b2a05cd971 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:16.903275 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aaf359-c6e9-42dc-a9e9-f7b2a05cd971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:16.903570 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aaf359-c6e9-42dc-a9e9-f7b2a05cd971 admin admin] PATH_INFO: `/v3/projects/d0865e5ed4d5448291ce53bbce1472b9/users/8d8b7e20cd4942eb8b834f79931c6981/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:16.921037 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aaf359-c6e9-42dc-a9e9-f7b2a05cd971 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0865e5ed4d5448291ce53bbce1472b9, user_id=8d8b7e20cd4942eb8b834f79931c6981, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:16.924206 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aaf359-c6e9-42dc-a9e9-f7b2a05cd971 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:16.957858 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffb423-c554-435c-b364-387c349aca97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:16.958505 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffb423-c554-435c-b364-387c349aca97 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:16.958798 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffb423-c554-435c-b364-387c349aca97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:16.959058 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffb423-c554-435c-b364-387c349aca97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:17.011809 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1340/2692] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:49:16 2022] PUT /identity/v3/projects/d0865e5ed4d5448291ce53bbce1472b9/users/8d8b7e20cd4942eb8b834f79931c6981/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:17.015770 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aaf359-c6e9-42dc-a9e9-f7b2a05cd971 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:17.018986 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ffb423-c554-435c-b364-387c349aca97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:17.020429 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ffb423-c554-435c-b364-387c349aca97 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:17.029682 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1353/2693] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:49:17.080226 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aaf359-c6e9-42dc-a9e9-f7b2a05cd971 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:17.085034 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e509f3ed-ea7f-405a-a03c-00b5a8db68c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:17.085841 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e509f3ed-ea7f-405a-a03c-00b5a8db68c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:17.086119 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e509f3ed-ea7f-405a-a03c-00b5a8db68c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:17.086388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e509f3ed-ea7f-405a-a03c-00b5a8db68c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:17.087045 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e509f3ed-ea7f-405a-a03c-00b5a8db68c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:17.088614 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e509f3ed-ea7f-405a-a03c-00b5a8db68c9 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:17.099244 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1341/2694] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:49:17.103961 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffb423-c554-435c-b364-387c349aca97 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:17.115408 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffb423-c554-435c-b364-387c349aca97 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:17.120523 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23623f-ff2b-4d81-832a-a1ac688485ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:17.121196 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23623f-ff2b-4d81-832a-a1ac688485ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:17.121479 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23623f-ff2b-4d81-832a-a1ac688485ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:17.121750 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23623f-ff2b-4d81-832a-a1ac688485ac admin admin] PATH_INFO: `/v3/projects/d0865e5ed4d5448291ce53bbce1472b9/users/8d8b7e20cd4942eb8b834f79931c6981/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:17.123765 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc23623f-ff2b-4d81-832a-a1ac688485ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0865e5ed4d5448291ce53bbce1472b9, user_id=8d8b7e20cd4942eb8b834f79931c6981, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:17.126600 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc23623f-ff2b-4d81-832a-a1ac688485ac admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:17.138031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1354/2695] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:49:17 2022] PUT /identity/v3/projects/d0865e5ed4d5448291ce53bbce1472b9/users/8d8b7e20cd4942eb8b834f79931c6981/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:17.148387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a36c32-cca4-46d5-92c0-2960b7309feb None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:17.148686 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a36c32-cca4-46d5-92c0-2960b7309feb None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:17.149197 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a36c32-cca4-46d5-92c0-2960b7309feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:17.177823 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-05a36c32-cca4-46d5-92c0-2960b7309feb None None] MFA Rules not processed for user `8d8b7e20cd4942eb8b834f79931c6981`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:17.179569 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-05a36c32-cca4-46d5-92c0-2960b7309feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:17.253173 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1342/2696] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:17 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:17.266087 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23623f-ff2b-4d81-832a-a1ac688485ac admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:17.267801 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-cc23623f-ff2b-4d81-832a-a1ac688485ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:17.292927 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:17.327754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23623f-ff2b-4d81-832a-a1ac688485ac admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:17.331792 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57789859-4b1e-4206-b825-c96004c59316 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:17.332386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57789859-4b1e-4206-b825-c96004c59316 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:17.332665 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57789859-4b1e-4206-b825-c96004c59316 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:17.332933 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57789859-4b1e-4206-b825-c96004c59316 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:17.342571 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57789859-4b1e-4206-b825-c96004c59316 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:17.344009 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57789859-4b1e-4206-b825-c96004c59316 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:17.354256 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1355/2697] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:17 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:49:17.356737 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:17.462490 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a36c32-cca4-46d5-92c0-2960b7309feb None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:17.472748 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a36c32-cca4-46d5-92c0-2960b7309feb None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:17.477073 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c05169-0055-471a-beb2-06c66226be57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:17.479372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c05169-0055-471a-beb2-06c66226be57 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:17.479372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c05169-0055-471a-beb2-06c66226be57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:17.479372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c05169-0055-471a-beb2-06c66226be57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:17.532864 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c05169-0055-471a-beb2-06c66226be57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:17.534255 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c05169-0055-471a-beb2-06c66226be57 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:17.543941 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1343/2698] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:49:18.028564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57789859-4b1e-4206-b825-c96004c59316 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:18.030014 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-57789859-4b1e-4206-b825-c96004c59316 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:18.093517 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57789859-4b1e-4206-b825-c96004c59316 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:18.098250 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7e6ae-d4bf-4114-9e6e-0a21ca052a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:18.098891 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7e6ae-d4bf-4114-9e6e-0a21ca052a18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:18.099171 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7e6ae-d4bf-4114-9e6e-0a21ca052a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:18.099444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7e6ae-d4bf-4114-9e6e-0a21ca052a18 admin admin] PATH_INFO: `/v3/users/b3ba063a311c458a8ef11424bea1c673` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:18.100587 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7e6ae-d4bf-4114-9e6e-0a21ca052a18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3ba063a311c458a8ef11424bea1c673)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:18.102534 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7e6ae-d4bf-4114-9e6e-0a21ca052a18 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:18.264921 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c05169-0055-471a-beb2-06c66226be57 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:18.266338 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e8c05169-0055-471a-beb2-06c66226be57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:18.281061 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c05169-0055-471a-beb2-06c66226be57 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:18.285835 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f3927-bab2-42ed-b87f-2d121bb9b2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:18.286209 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f3927-bab2-42ed-b87f-2d121bb9b2cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:18.286486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f3927-bab2-42ed-b87f-2d121bb9b2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:18.286753 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f3927-bab2-42ed-b87f-2d121bb9b2cc admin admin] PATH_INFO: `/v3/users/137b0282018e4268a2c245e945155235` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:18.288535 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099f3927-bab2-42ed-b87f-2d121bb9b2cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=137b0282018e4268a2c245e945155235)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:18.290463 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099f3927-bab2-42ed-b87f-2d121bb9b2cc admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:18.394958 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0fa7e6ae-d4bf-4114-9e6e-0a21ca052a18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3ba063a311c458a8ef11424bea1c673'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:18.398107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0fa7e6ae-d4bf-4114-9e6e-0a21ca052a18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3ba063a311c458a8ef11424bea1c673'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:18.398946 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0fa7e6ae-d4bf-4114-9e6e-0a21ca052a18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3ba063a311c458a8ef11424bea1c673'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:18.415398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0fa7e6ae-d4bf-4114-9e6e-0a21ca052a18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3ba063a311c458a8ef11424bea1c673'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:18.493661 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-0fa7e6ae-d4bf-4114-9e6e-0a21ca052a18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3ba063a311c458a8ef11424bea1c673'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:18.502289 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:18.509651 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-099f3927-bab2-42ed-b87f-2d121bb9b2cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '137b0282018e4268a2c245e945155235'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:18.510658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-099f3927-bab2-42ed-b87f-2d121bb9b2cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '137b0282018e4268a2c245e945155235'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:18.511264 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-099f3927-bab2-42ed-b87f-2d121bb9b2cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '137b0282018e4268a2c245e945155235'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:18.514369 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1356/2699] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:49:18 2022] DELETE /identity/v3/users/b3ba063a311c458a8ef11424bea1c673 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:18.518503 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff722ef-68f6-4202-b5d4-fff741c457fa None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:18.518795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff722ef-68f6-4202-b5d4-fff741c457fa None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:18.519069 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff722ef-68f6-4202-b5d4-fff741c457fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:18.520005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-099f3927-bab2-42ed-b87f-2d121bb9b2cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '137b0282018e4268a2c245e945155235'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:18.523089 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:18.538258 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-2ff722ef-68f6-4202-b5d4-fff741c457fa None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:18.539498 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2ff722ef-68f6-4202-b5d4-fff741c457fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:18.579105 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-099f3927-bab2-42ed-b87f-2d121bb9b2cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '137b0282018e4268a2c245e945155235'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:18.597833 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1344/2700] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:49:18 2022] DELETE /identity/v3/users/137b0282018e4268a2c245e945155235 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:49:18.598442 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1357/2701] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:49:18.602413 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48224e7-981b-4343-9320-972996542d3d None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:18.602702 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48224e7-981b-4343-9320-972996542d3d None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:18.602976 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48224e7-981b-4343-9320-972996542d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:18.608891 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff722ef-68f6-4202-b5d4-fff741c457fa None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:18.610258 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-2ff722ef-68f6-4202-b5d4-fff741c457fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:18.622616 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-e48224e7-981b-4343-9320-972996542d3d None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:18.623806 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e48224e7-981b-4343-9320-972996542d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:18.675819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff722ef-68f6-4202-b5d4-fff741c457fa None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:18.690818 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1345/2702] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:49:18.701087 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48224e7-981b-4343-9320-972996542d3d None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:18.726664 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae032d5-a218-4195-a275-d12a6aa42adf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:18.727287 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae032d5-a218-4195-a275-d12a6aa42adf service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:18.729249 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae032d5-a218-4195-a275-d12a6aa42adf service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:18.729524 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae032d5-a218-4195-a275-d12a6aa42adf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:18.745105 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae032d5-a218-4195-a275-d12a6aa42adf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:18.746664 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae032d5-a218-4195-a275-d12a6aa42adf service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:18.758661 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48224e7-981b-4343-9320-972996542d3d None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:18.759310 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1358/2703] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:49:18.767177 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a99be2-932c-48ed-a0ec-dd821e3b6a3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:18.771366 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a99be2-932c-48ed-a0ec-dd821e3b6a3e service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:18.771366 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a99be2-932c-48ed-a0ec-dd821e3b6a3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:18.771366 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a99be2-932c-48ed-a0ec-dd821e3b6a3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:18.779819 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a99be2-932c-48ed-a0ec-dd821e3b6a3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:18.781896 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a99be2-932c-48ed-a0ec-dd821e3b6a3e service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:18.805502 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1346/2704] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:49:18.807755 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd2d12-fd60-4bef-8d6b-6f713da0d0e5 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:18.808037 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd2d12-fd60-4bef-8d6b-6f713da0d0e5 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:18.808307 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd2d12-fd60-4bef-8d6b-6f713da0d0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:18.827254 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a7cd2d12-fd60-4bef-8d6b-6f713da0d0e5 None None] MFA Rules not processed for user `d89b729776114f7ab97c7a99f5a777c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:18.828452 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a7cd2d12-fd60-4bef-8d6b-6f713da0d0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:18.886178 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1359/2705] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:18.894763 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:19.410286 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a99be2-932c-48ed-a0ec-dd821e3b6a3e service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:19.412221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e6a99be2-932c-48ed-a0ec-dd821e3b6a3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:19.436360 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a99be2-932c-48ed-a0ec-dd821e3b6a3e service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:19.440726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd2d12-fd60-4bef-8d6b-6f713da0d0e5 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:19.441482 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5afd1f4-3d78-4b66-8557-1c5e1a3225e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:19.442210 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afd1f4-3d78-4b66-8557-1c5e1a3225e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:19.442561 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afd1f4-3d78-4b66-8557-1c5e1a3225e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:19.442909 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afd1f4-3d78-4b66-8557-1c5e1a3225e0 admin admin] PATH_INFO: `/v3/projects/718f988579854efe8ff0609ff2682e05` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:19.443820 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a7cd2d12-fd60-4bef-8d6b-6f713da0d0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:19.444715 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5afd1f4-3d78-4b66-8557-1c5e1a3225e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=718f988579854efe8ff0609ff2682e05)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:19.447347 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5afd1f4-3d78-4b66-8557-1c5e1a3225e0 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:19.476591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd2d12-fd60-4bef-8d6b-6f713da0d0e5 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:19.489240 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12f0fd-ca22-4747-ae04-4798a1316e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:19.490057 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12f0fd-ca22-4747-ae04-4798a1316e9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:19.490199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12f0fd-ca22-4747-ae04-4798a1316e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:19.490469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12f0fd-ca22-4747-ae04-4798a1316e9f admin admin] PATH_INFO: `/v3/projects/e32e3d331f154cd08d5e6d1e6c9225ee` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:19.492303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12f0fd-ca22-4747-ae04-4798a1316e9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e32e3d331f154cd08d5e6d1e6c9225ee)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:19.494320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12f0fd-ca22-4747-ae04-4798a1316e9f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:19.583032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-b5afd1f4-3d78-4b66-8557-1c5e1a3225e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '718f988579854efe8ff0609ff2682e05', 'request_id': 'req-b5afd1f4-3d78-4b66-8557-1c5e1a3225e0'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:19.590021 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-b5afd1f4-3d78-4b66-8557-1c5e1a3225e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '718f988579854efe8ff0609ff2682e05', 'request_id': 'req-b5afd1f4-3d78-4b66-8557-1c5e1a3225e0'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:19.596663 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-b5afd1f4-3d78-4b66-8557-1c5e1a3225e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '718f988579854efe8ff0609ff2682e05', 'request_id': 'req-b5afd1f4-3d78-4b66-8557-1c5e1a3225e0'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:19.682652 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8d12f0fd-ca22-4747-ae04-4798a1316e9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e32e3d331f154cd08d5e6d1e6c9225ee', 'request_id': 'req-8d12f0fd-ca22-4747-ae04-4798a1316e9f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:19.690198 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:19.694936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-b5afd1f4-3d78-4b66-8557-1c5e1a3225e0 admin admin] The token cache is being invalidate because project 718f988579854efe8ff0609ff2682e05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:49:19.697524 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-b5afd1f4-3d78-4b66-8557-1c5e1a3225e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:19.697524 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-b5afd1f4-3d78-4b66-8557-1c5e1a3225e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:19.697980 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1347/2706] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:49:19 2022] DELETE /identity/v3/projects/718f988579854efe8ff0609ff2682e05 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:19.701878 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5afd1f4-3d78-4b66-8557-1c5e1a3225e0 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:19.703357 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b5afd1f4-3d78-4b66-8557-1c5e1a3225e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:19.771718 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5afd1f4-3d78-4b66-8557-1c5e1a3225e0 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:19.790401 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8d12f0fd-ca22-4747-ae04-4798a1316e9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e32e3d331f154cd08d5e6d1e6c9225ee', 'request_id': 'req-8d12f0fd-ca22-4747-ae04-4798a1316e9f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:19.797723 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8d12f0fd-ca22-4747-ae04-4798a1316e9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e32e3d331f154cd08d5e6d1e6c9225ee', 'request_id': 'req-8d12f0fd-ca22-4747-ae04-4798a1316e9f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:19.811202 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8d12f0fd-ca22-4747-ae04-4798a1316e9f admin admin] The token cache is being invalidate because project e32e3d331f154cd08d5e6d1e6c9225ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:49:19.811489 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8d12f0fd-ca22-4747-ae04-4798a1316e9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:19.812327 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8d12f0fd-ca22-4747-ae04-4798a1316e9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:19.817290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1360/2707] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:49:19 2022] DELETE /identity/v3/projects/e32e3d331f154cd08d5e6d1e6c9225ee => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:19.818235 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12f0fd-ca22-4747-ae04-4798a1316e9f admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:19.819923 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8d12f0fd-ca22-4747-ae04-4798a1316e9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:19.846052 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45ab28-7557-40de-9b26-373549398689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:19.846748 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45ab28-7557-40de-9b26-373549398689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:19.847026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45ab28-7557-40de-9b26-373549398689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:19.847292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45ab28-7557-40de-9b26-373549398689 admin admin] PATH_INFO: `/v3/users/d5ee1b1414fa468cb55bd1d9143a8e28` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:19.848457 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be45ab28-7557-40de-9b26-373549398689 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5ee1b1414fa468cb55bd1d9143a8e28)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:19.851200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be45ab28-7557-40de-9b26-373549398689 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:19.918226 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12f0fd-ca22-4747-ae04-4798a1316e9f admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:19.923170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce378c5b-1853-4095-8346-7b88649128df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:19.923814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce378c5b-1853-4095-8346-7b88649128df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:19.924140 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce378c5b-1853-4095-8346-7b88649128df admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:19.924415 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce378c5b-1853-4095-8346-7b88649128df admin admin] PATH_INFO: `/v3/users/879cdba023bc454db24418e04179b82f` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:19.925755 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce378c5b-1853-4095-8346-7b88649128df admin admin] RBAC: Authorizing `identity:delete_user(user_id=879cdba023bc454db24418e04179b82f)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:19.927646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce378c5b-1853-4095-8346-7b88649128df admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:20.112339 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-be45ab28-7557-40de-9b26-373549398689 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5ee1b1414fa468cb55bd1d9143a8e28'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:20.112992 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-be45ab28-7557-40de-9b26-373549398689 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5ee1b1414fa468cb55bd1d9143a8e28'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:20.113627 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-be45ab28-7557-40de-9b26-373549398689 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5ee1b1414fa468cb55bd1d9143a8e28'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:20.121043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-be45ab28-7557-40de-9b26-373549398689 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5ee1b1414fa468cb55bd1d9143a8e28'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:20.146253 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ce378c5b-1853-4095-8346-7b88649128df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '879cdba023bc454db24418e04179b82f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:20.147589 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ce378c5b-1853-4095-8346-7b88649128df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '879cdba023bc454db24418e04179b82f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:20.148464 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ce378c5b-1853-4095-8346-7b88649128df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '879cdba023bc454db24418e04179b82f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:20.155841 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ce378c5b-1853-4095-8346-7b88649128df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '879cdba023bc454db24418e04179b82f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:20.224582 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-be45ab28-7557-40de-9b26-373549398689 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5ee1b1414fa468cb55bd1d9143a8e28'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:20.242967 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1348/2708] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:49:19 2022] DELETE /identity/v3/users/d5ee1b1414fa468cb55bd1d9143a8e28 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:49:20.281875 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ce378c5b-1853-4095-8346-7b88649128df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '879cdba023bc454db24418e04179b82f'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:20.296221 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:20.303470 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1361/2709] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:49:19 2022] DELETE /identity/v3/users/879cdba023bc454db24418e04179b82f => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:20.707337 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45ab28-7557-40de-9b26-373549398689 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:20.708835 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-be45ab28-7557-40de-9b26-373549398689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:20.733478 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce378c5b-1853-4095-8346-7b88649128df admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:20.735499 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ce378c5b-1853-4095-8346-7b88649128df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:20.789958 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45ab28-7557-40de-9b26-373549398689 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:20.825344 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce378c5b-1853-4095-8346-7b88649128df admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:20.830042 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a216905c-a271-4b95-ac4a-e681a39d1ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:20.830666 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216905c-a271-4b95-ac4a-e681a39d1ec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:20.830939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216905c-a271-4b95-ac4a-e681a39d1ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:20.831204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216905c-a271-4b95-ac4a-e681a39d1ec1 admin admin] PATH_INFO: `/v3/projects/9ac2dfc942b9423a97d96353f595e5c0` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:20.832470 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a216905c-a271-4b95-ac4a-e681a39d1ec1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ac2dfc942b9423a97d96353f595e5c0)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:20.834494 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a216905c-a271-4b95-ac4a-e681a39d1ec1 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:20.848994 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b07145e-5f7e-4644-a198-66ccfba25847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:20.849652 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07145e-5f7e-4644-a198-66ccfba25847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:20.850596 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07145e-5f7e-4644-a198-66ccfba25847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:20.850702 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07145e-5f7e-4644-a198-66ccfba25847 admin admin] PATH_INFO: `/v3/projects/ba3bb0c70dcc440287cf93d962ea6795` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:20.851802 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b07145e-5f7e-4644-a198-66ccfba25847 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba3bb0c70dcc440287cf93d962ea6795)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:20.854088 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b07145e-5f7e-4644-a198-66ccfba25847 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:20.871879 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:20.996339 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a216905c-a271-4b95-ac4a-e681a39d1ec1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ac2dfc942b9423a97d96353f595e5c0', 'request_id': 'req-a216905c-a271-4b95-ac4a-e681a39d1ec1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:21.082050 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a216905c-a271-4b95-ac4a-e681a39d1ec1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ac2dfc942b9423a97d96353f595e5c0', 'request_id': 'req-a216905c-a271-4b95-ac4a-e681a39d1ec1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:21.089265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a216905c-a271-4b95-ac4a-e681a39d1ec1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ac2dfc942b9423a97d96353f595e5c0', 'request_id': 'req-a216905c-a271-4b95-ac4a-e681a39d1ec1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:21.101096 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a216905c-a271-4b95-ac4a-e681a39d1ec1 admin admin] The token cache is being invalidate because project 9ac2dfc942b9423a97d96353f595e5c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:49:21.101502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a216905c-a271-4b95-ac4a-e681a39d1ec1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:21.102220 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-a216905c-a271-4b95-ac4a-e681a39d1ec1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:21.103965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1362/2710] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:49:20 2022] DELETE /identity/v3/projects/9ac2dfc942b9423a97d96353f595e5c0 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:21.108073 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a216905c-a271-4b95-ac4a-e681a39d1ec1 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:21.109450 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a216905c-a271-4b95-ac4a-e681a39d1ec1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:21.136995 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1b07145e-5f7e-4644-a198-66ccfba25847 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba3bb0c70dcc440287cf93d962ea6795', 'request_id': 'req-1b07145e-5f7e-4644-a198-66ccfba25847'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:21.143933 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1b07145e-5f7e-4644-a198-66ccfba25847 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba3bb0c70dcc440287cf93d962ea6795', 'request_id': 'req-1b07145e-5f7e-4644-a198-66ccfba25847'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:21.150327 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1b07145e-5f7e-4644-a198-66ccfba25847 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba3bb0c70dcc440287cf93d962ea6795', 'request_id': 'req-1b07145e-5f7e-4644-a198-66ccfba25847'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:21.174711 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a216905c-a271-4b95-ac4a-e681a39d1ec1 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:21.221575 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1b07145e-5f7e-4644-a198-66ccfba25847 admin admin] The token cache is being invalidate because project ba3bb0c70dcc440287cf93d962ea6795 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:49:21.221723 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1b07145e-5f7e-4644-a198-66ccfba25847 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:21.222780 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-1b07145e-5f7e-4644-a198-66ccfba25847 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:21.230863 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1349/2711] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:49:20 2022] DELETE /identity/v3/projects/ba3bb0c70dcc440287cf93d962ea6795 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:21.232796 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e7a40-0f67-43e9-8715-69cc79ddffb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:21.233475 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e7a40-0f67-43e9-8715-69cc79ddffb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:21.233628 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7daf43-a623-4690-ba9d-476413d3305a None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:21.233740 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e7a40-0f67-43e9-8715-69cc79ddffb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:21.233834 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7daf43-a623-4690-ba9d-476413d3305a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:21.234002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e7a40-0f67-43e9-8715-69cc79ddffb7 admin admin] PATH_INFO: `/v3/users/486a6618f43b4f09b70db66dc9513f3b` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:21.234107 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7daf43-a623-4690-ba9d-476413d3305a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:21.235216 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e7a40-0f67-43e9-8715-69cc79ddffb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=486a6618f43b4f09b70db66dc9513f3b)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:21.237262 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e7a40-0f67-43e9-8715-69cc79ddffb7 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:21.256071 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-1a7daf43-a623-4690-ba9d-476413d3305a None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:21.256071 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1a7daf43-a623-4690-ba9d-476413d3305a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:21.263842 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1350/2712] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:21.271174 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52780524-9a5d-4334-8709-7ce6e5b8e706 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:21.271487 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52780524-9a5d-4334-8709-7ce6e5b8e706 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:21.272297 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52780524-9a5d-4334-8709-7ce6e5b8e706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:21.291533 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-52780524-9a5d-4334-8709-7ce6e5b8e706 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:21.292716 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-52780524-9a5d-4334-8709-7ce6e5b8e706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:21.300486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1351/2713] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:49:21.304662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52780524-9a5d-4334-8709-7ce6e5b8e706 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:21.313356 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52780524-9a5d-4334-8709-7ce6e5b8e706 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:21.317852 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b4d01-5fe2-4e6e-ac05-0feff1d2d50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:21.318417 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b4d01-5fe2-4e6e-ac05-0feff1d2d50a admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:21.319228 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b4d01-5fe2-4e6e-ac05-0feff1d2d50a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:21.319497 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b4d01-5fe2-4e6e-ac05-0feff1d2d50a admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:21.320135 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b4d01-5fe2-4e6e-ac05-0feff1d2d50a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:49:21.320428 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b4d01-5fe2-4e6e-ac05-0feff1d2d50a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:21.322006 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b4d01-5fe2-4e6e-ac05-0feff1d2d50a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:21.334119 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1352/2714] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:49:21.339211 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b4d01-5fe2-4e6e-ac05-0feff1d2d50a admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:21.349014 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b4d01-5fe2-4e6e-ac05-0feff1d2d50a admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:21.354601 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17b923-6d26-4e7b-b943-a07bda4ecb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:21.355191 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17b923-6d26-4e7b-b943-a07bda4ecb5b admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:21.355468 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17b923-6d26-4e7b-b943-a07bda4ecb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:21.355736 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17b923-6d26-4e7b-b943-a07bda4ecb5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:21.356351 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f17b923-6d26-4e7b-b943-a07bda4ecb5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:21.358249 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f17b923-6d26-4e7b-b943-a07bda4ecb5b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:21.416458 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-981e7a40-0f67-43e9-8715-69cc79ddffb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '486a6618f43b4f09b70db66dc9513f3b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:21.417120 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-981e7a40-0f67-43e9-8715-69cc79ddffb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '486a6618f43b4f09b70db66dc9513f3b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:21.417882 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-981e7a40-0f67-43e9-8715-69cc79ddffb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '486a6618f43b4f09b70db66dc9513f3b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:21.425006 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-981e7a40-0f67-43e9-8715-69cc79ddffb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '486a6618f43b4f09b70db66dc9513f3b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:21.438420 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1353/2715] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:49:21 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:49:21.442546 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17b923-6d26-4e7b-b943-a07bda4ecb5b admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:21.444439 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5f17b923-6d26-4e7b-b943-a07bda4ecb5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:21.507514 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17b923-6d26-4e7b-b943-a07bda4ecb5b admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:21.512860 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370eaa7-4d32-481e-a3ba-231ae0224009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:21.513545 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370eaa7-4d32-481e-a3ba-231ae0224009 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:21.513820 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370eaa7-4d32-481e-a3ba-231ae0224009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:21.514087 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370eaa7-4d32-481e-a3ba-231ae0224009 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:21.514718 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a370eaa7-4d32-481e-a3ba-231ae0224009 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:21.516611 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a370eaa7-4d32-481e-a3ba-231ae0224009 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:21.521362 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-981e7a40-0f67-43e9-8715-69cc79ddffb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '486a6618f43b4f09b70db66dc9513f3b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:21.540195 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1363/2716] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:49:21 2022] DELETE /identity/v3/users/486a6618f43b4f09b70db66dc9513f3b => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:21.541312 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:21.619431 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1354/2717] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:49:21 2022] POST /identity/v3/users => generated 283 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:49:21.623571 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e7a40-0f67-43e9-8715-69cc79ddffb7 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:21.685418 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e7a40-0f67-43e9-8715-69cc79ddffb7 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:21.690215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f29eed-dd49-484c-8cbe-479fe3dd5298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:21.690819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f29eed-dd49-484c-8cbe-479fe3dd5298 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:21.691152 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f29eed-dd49-484c-8cbe-479fe3dd5298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:21.691422 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f29eed-dd49-484c-8cbe-479fe3dd5298 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:21.692066 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f29eed-dd49-484c-8cbe-479fe3dd5298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:21.694410 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f29eed-dd49-484c-8cbe-479fe3dd5298 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:21.704490 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1364/2718] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:49:21.708308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370eaa7-4d32-481e-a3ba-231ae0224009 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:21.719492 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370eaa7-4d32-481e-a3ba-231ae0224009 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:21.724801 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb74e7-1abd-41f4-8677-0d96408d7420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:21.725472 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb74e7-1abd-41f4-8677-0d96408d7420 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:21.725749 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb74e7-1abd-41f4-8677-0d96408d7420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:21.726066 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb74e7-1abd-41f4-8677-0d96408d7420 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:21.726718 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcb74e7-1abd-41f4-8677-0d96408d7420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:21.728377 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcb74e7-1abd-41f4-8677-0d96408d7420 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:21.742558 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1355/2719] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:49:21.744533 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f29eed-dd49-484c-8cbe-479fe3dd5298 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:21.754401 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f29eed-dd49-484c-8cbe-479fe3dd5298 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:21.760873 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f115af-a403-47ea-a934-0007b2b3cab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:21.761483 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f115af-a403-47ea-a934-0007b2b3cab6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:21.761889 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f115af-a403-47ea-a934-0007b2b3cab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:21.762167 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f115af-a403-47ea-a934-0007b2b3cab6 admin admin] PATH_INFO: `/v3/projects/b7d1348f5958499b9812242ed59ce2ee/users/fa37eee3ff804505a2fa0891193a2c52/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:21.779302 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f115af-a403-47ea-a934-0007b2b3cab6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7d1348f5958499b9812242ed59ce2ee, user_id=fa37eee3ff804505a2fa0891193a2c52, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:21.782061 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f115af-a403-47ea-a934-0007b2b3cab6 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:21.843902 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb74e7-1abd-41f4-8677-0d96408d7420 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:21.845778 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8dcb74e7-1abd-41f4-8677-0d96408d7420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:21.863201 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb74e7-1abd-41f4-8677-0d96408d7420 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:21.865060 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1365/2720] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:49:21 2022] PUT /identity/v3/projects/b7d1348f5958499b9812242ed59ce2ee/users/fa37eee3ff804505a2fa0891193a2c52/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:21.879137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cadf81-b5f6-4142-b06c-d75e976b9080 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:21.879607 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cadf81-b5f6-4142-b06c-d75e976b9080 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:21.880040 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cadf81-b5f6-4142-b06c-d75e976b9080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:21.920966 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d4cadf81-b5f6-4142-b06c-d75e976b9080 None None] MFA Rules not processed for user `fa37eee3ff804505a2fa0891193a2c52`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:21.922203 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d4cadf81-b5f6-4142-b06c-d75e976b9080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:21.925745 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba2ef6-e41c-4733-89d4-239d11281a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:21.926959 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba2ef6-e41c-4733-89d4-239d11281a0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:21.927561 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba2ef6-e41c-4733-89d4-239d11281a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:21.928883 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba2ef6-e41c-4733-89d4-239d11281a0a admin admin] PATH_INFO: `/v3/users/5db25d6359594956b721f6a6525cec96` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:21.930361 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba2ef6-e41c-4733-89d4-239d11281a0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5db25d6359594956b721f6a6525cec96)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:21.932415 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba2ef6-e41c-4733-89d4-239d11281a0a admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:21.986169 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1366/2721] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:49:21.997801 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cadf81-b5f6-4142-b06c-d75e976b9080 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:21.999139 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d4cadf81-b5f6-4142-b06c-d75e976b9080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:22.059975 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cadf81-b5f6-4142-b06c-d75e976b9080 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:22.063682 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a8711-90e8-45f1-a516-2ab3cc8fb7b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:22.064428 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a8711-90e8-45f1-a516-2ab3cc8fb7b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:22.064711 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a8711-90e8-45f1-a516-2ab3cc8fb7b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:22.065034 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a8711-90e8-45f1-a516-2ab3cc8fb7b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:22.075418 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a8711-90e8-45f1-a516-2ab3cc8fb7b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:22.076915 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a8711-90e8-45f1-a516-2ab3cc8fb7b0 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:22.082971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:22.085368 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-89ba2ef6-e41c-4733-89d4-239d11281a0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5db25d6359594956b721f6a6525cec96'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:22.086212 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-89ba2ef6-e41c-4733-89d4-239d11281a0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5db25d6359594956b721f6a6525cec96'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:22.086971 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-89ba2ef6-e41c-4733-89d4-239d11281a0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5db25d6359594956b721f6a6525cec96'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:22.089586 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1367/2722] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:49:22.089922 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a8711-90e8-45f1-a516-2ab3cc8fb7b0 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:22.092062 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-552a8711-90e8-45f1-a516-2ab3cc8fb7b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:22.107618 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a8711-90e8-45f1-a516-2ab3cc8fb7b0 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:22.111362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-89ba2ef6-e41c-4733-89d4-239d11281a0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5db25d6359594956b721f6a6525cec96'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:22.113784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff33c6-5980-4f37-ab7b-115d14a7c1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:22.114159 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff33c6-5980-4f37-ab7b-115d14a7c1f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:22.116764 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff33c6-5980-4f37-ab7b-115d14a7c1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:22.116764 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff33c6-5980-4f37-ab7b-115d14a7c1f4 admin admin] PATH_INFO: `/v3/projects/ff0854d3616747c18af8c8de206630c8` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:22.117846 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff33c6-5980-4f37-ab7b-115d14a7c1f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff0854d3616747c18af8c8de206630c8)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:22.119991 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff33c6-5980-4f37-ab7b-115d14a7c1f4 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:22.196042 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-89ba2ef6-e41c-4733-89d4-239d11281a0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5db25d6359594956b721f6a6525cec96'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:22.218623 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1356/2723] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:49:21 2022] DELETE /identity/v3/users/5db25d6359594956b721f6a6525cec96 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:22.223206 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0e8ab-a315-47fb-9470-4aa6c2659366 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:22.223487 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0e8ab-a315-47fb-9470-4aa6c2659366 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:22.223766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0e8ab-a315-47fb-9470-4aa6c2659366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:22.246338 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-79b0e8ab-a315-47fb-9470-4aa6c2659366 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:22.248119 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-79b0e8ab-a315-47fb-9470-4aa6c2659366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:22.252443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e1ff33c6-5980-4f37-ab7b-115d14a7c1f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff0854d3616747c18af8c8de206630c8', 'request_id': 'req-e1ff33c6-5980-4f37-ab7b-115d14a7c1f4'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:22.309122 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1357/2724] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:49:22.318920 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0e8ab-a315-47fb-9470-4aa6c2659366 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:22.320364 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-79b0e8ab-a315-47fb-9470-4aa6c2659366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:22.327397 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e1ff33c6-5980-4f37-ab7b-115d14a7c1f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff0854d3616747c18af8c8de206630c8', 'request_id': 'req-e1ff33c6-5980-4f37-ab7b-115d14a7c1f4'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:22.334863 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e1ff33c6-5980-4f37-ab7b-115d14a7c1f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff0854d3616747c18af8c8de206630c8', 'request_id': 'req-e1ff33c6-5980-4f37-ab7b-115d14a7c1f4'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:22.348045 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e1ff33c6-5980-4f37-ab7b-115d14a7c1f4 admin admin] The token cache is being invalidate because project ff0854d3616747c18af8c8de206630c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:49:22.348329 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e1ff33c6-5980-4f37-ab7b-115d14a7c1f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:22.351613 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-e1ff33c6-5980-4f37-ab7b-115d14a7c1f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:22.351613 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1368/2725] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:49:22 2022] DELETE /identity/v3/projects/ff0854d3616747c18af8c8de206630c8 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:22.358673 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019fe84-6811-4f68-ba52-d453150179cc None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:22.358951 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019fe84-6811-4f68-ba52-d453150179cc None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:22.359223 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019fe84-6811-4f68-ba52-d453150179cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:22.383293 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0e8ab-a315-47fb-9470-4aa6c2659366 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:22.389341 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bdd8ba-8f7e-4cf0-93ac-d8bc6334a571 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:22.390586 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bdd8ba-8f7e-4cf0-93ac-d8bc6334a571 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:22.390871 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bdd8ba-8f7e-4cf0-93ac-d8bc6334a571 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:22.391139 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bdd8ba-8f7e-4cf0-93ac-d8bc6334a571 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:22.392348 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b019fe84-6811-4f68-ba52-d453150179cc None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:22.392923 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-90bdd8ba-8f7e-4cf0-93ac-d8bc6334a571 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:22.393750 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b019fe84-6811-4f68-ba52-d453150179cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:22.403181 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1369/2726] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:22.406338 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bdd8ba-8f7e-4cf0-93ac-d8bc6334a571 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:22.407829 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bdd8ba-8f7e-4cf0-93ac-d8bc6334a571 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:22.414880 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933e209-4373-4da9-8497-c961bbcceb53 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:22.415280 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933e209-4373-4da9-8497-c961bbcceb53 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:22.415813 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933e209-4373-4da9-8497-c961bbcceb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:22.422164 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1358/2727] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:49:22.436677 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-3933e209-4373-4da9-8497-c961bbcceb53 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:22.437997 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3933e209-4373-4da9-8497-c961bbcceb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:22.449942 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1370/2728] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:22.454895 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:22.458459 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bdd8ba-8f7e-4cf0-93ac-d8bc6334a571 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:22.471690 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bdd8ba-8f7e-4cf0-93ac-d8bc6334a571 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:22.483215 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec0883-a215-46b3-8d8b-920471924b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:22.484106 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec0883-a215-46b3-8d8b-920471924b42 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:22.484378 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec0883-a215-46b3-8d8b-920471924b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:22.484687 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec0883-a215-46b3-8d8b-920471924b42 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:22.485383 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec0883-a215-46b3-8d8b-920471924b42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:49:22.485839 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec0883-a215-46b3-8d8b-920471924b42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:22.487497 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec0883-a215-46b3-8d8b-920471924b42 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:22.502016 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1359/2729] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:49:22.506601 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3933e209-4373-4da9-8497-c961bbcceb53 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:22.516203 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3933e209-4373-4da9-8497-c961bbcceb53 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:22.521013 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22759134-25fc-4f95-af17-b35e243d8523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:22.522197 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22759134-25fc-4f95-af17-b35e243d8523 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:22.522469 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22759134-25fc-4f95-af17-b35e243d8523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:22.522748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22759134-25fc-4f95-af17-b35e243d8523 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:22.523352 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22759134-25fc-4f95-af17-b35e243d8523 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:22.525285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22759134-25fc-4f95-af17-b35e243d8523 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:22.641678 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1371/2730] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:49:22 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:49:22.642853 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec0883-a215-46b3-8d8b-920471924b42 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:22.654708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec0883-a215-46b3-8d8b-920471924b42 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:22.740852 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1d5b4-aedc-47b4-9290-50bb0071f47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:22.742087 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1d5b4-aedc-47b4-9290-50bb0071f47d admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:22.742465 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1d5b4-aedc-47b4-9290-50bb0071f47d admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:22.742846 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1d5b4-aedc-47b4-9290-50bb0071f47d admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:22.743614 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e1d5b4-aedc-47b4-9290-50bb0071f47d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:22.746360 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e1d5b4-aedc-47b4-9290-50bb0071f47d admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:22.878901 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:22.887757 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1360/2731] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:49:22 2022] POST /identity/v3/users => generated 292 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:49:22.888990 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22759134-25fc-4f95-af17-b35e243d8523 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:22.901718 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22759134-25fc-4f95-af17-b35e243d8523 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:22.906756 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d56c35-92ba-44c7-8979-7082a3529008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:22.907459 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d56c35-92ba-44c7-8979-7082a3529008 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:22.907855 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d56c35-92ba-44c7-8979-7082a3529008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:22.908128 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d56c35-92ba-44c7-8979-7082a3529008 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:22.908766 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d56c35-92ba-44c7-8979-7082a3529008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:22.910485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d56c35-92ba-44c7-8979-7082a3529008 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:22.920825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1372/2732] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:49:22.924789 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1d5b4-aedc-47b4-9290-50bb0071f47d admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:22.934767 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1d5b4-aedc-47b4-9290-50bb0071f47d admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:22.939364 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de43b5f5-29be-4df5-8df8-544240db4045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:22.942870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43b5f5-29be-4df5-8df8-544240db4045 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:22.942870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43b5f5-29be-4df5-8df8-544240db4045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:22.942870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43b5f5-29be-4df5-8df8-544240db4045 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:22.942870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de43b5f5-29be-4df5-8df8-544240db4045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:22.942870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de43b5f5-29be-4df5-8df8-544240db4045 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:22.954165 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1361/2733] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:49:22.958427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d56c35-92ba-44c7-8979-7082a3529008 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:22.969441 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d56c35-92ba-44c7-8979-7082a3529008 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:22.973958 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed764447-a7c3-4a5b-aea7-0c0e6c771ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:22.974604 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed764447-a7c3-4a5b-aea7-0c0e6c771ce4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:22.975082 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed764447-a7c3-4a5b-aea7-0c0e6c771ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:22.975361 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed764447-a7c3-4a5b-aea7-0c0e6c771ce4 admin admin] PATH_INFO: `/v3/projects/0bbf785f7de94ef7ad423568d41af122/users/891658773e7546c78c3213a3f5233fd4/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:22.990125 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed764447-a7c3-4a5b-aea7-0c0e6c771ce4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bbf785f7de94ef7ad423568d41af122, user_id=891658773e7546c78c3213a3f5233fd4, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:22.993116 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed764447-a7c3-4a5b-aea7-0c0e6c771ce4 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:23.107329 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1373/2734] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:49:22 2022] PUT /identity/v3/projects/0bbf785f7de94ef7ad423568d41af122/users/891658773e7546c78c3213a3f5233fd4/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:23.112457 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15cdf3b-489c-4096-a0c8-9521187aa2dd None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:23.112751 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15cdf3b-489c-4096-a0c8-9521187aa2dd None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:23.113032 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15cdf3b-489c-4096-a0c8-9521187aa2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:23.133266 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-c15cdf3b-489c-4096-a0c8-9521187aa2dd None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:23.134484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c15cdf3b-489c-4096-a0c8-9521187aa2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:23.198585 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1362/2735] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:23.208874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed764447-a7c3-4a5b-aea7-0c0e6c771ce4 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:23.211114 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ed764447-a7c3-4a5b-aea7-0c0e6c771ce4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:23.265043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15cdf3b-489c-4096-a0c8-9521187aa2dd None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:23.271319 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c15cdf3b-489c-4096-a0c8-9521187aa2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:23.286922 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed764447-a7c3-4a5b-aea7-0c0e6c771ce4 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:23.290976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3569931-245b-475c-8990-a257d9caed26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:23.291654 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3569931-245b-475c-8990-a257d9caed26 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:23.291944 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3569931-245b-475c-8990-a257d9caed26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:23.292220 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3569931-245b-475c-8990-a257d9caed26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:23.297482 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15cdf3b-489c-4096-a0c8-9521187aa2dd None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:23.303729 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3569931-245b-475c-8990-a257d9caed26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:23.304203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069884c-5590-4948-8d85-a3e4e53c526c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:23.309836 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069884c-5590-4948-8d85-a3e4e53c526c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:23.309836 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069884c-5590-4948-8d85-a3e4e53c526c admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:23.309836 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069884c-5590-4948-8d85-a3e4e53c526c admin admin] PATH_INFO: `/v3/projects/2216c5cb9f184935960b2d3880aaf25f` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:23.310335 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3569931-245b-475c-8990-a257d9caed26 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:23.310421 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069884c-5590-4948-8d85-a3e4e53c526c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2216c5cb9f184935960b2d3880aaf25f)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:23.310421 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069884c-5590-4948-8d85-a3e4e53c526c admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:23.315991 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1374/2736] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:49:23.419848 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:23.477405 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-2069884c-5590-4948-8d85-a3e4e53c526c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2216c5cb9f184935960b2d3880aaf25f', 'request_id': 'req-2069884c-5590-4948-8d85-a3e4e53c526c'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:23.484293 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-2069884c-5590-4948-8d85-a3e4e53c526c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2216c5cb9f184935960b2d3880aaf25f', 'request_id': 'req-2069884c-5590-4948-8d85-a3e4e53c526c'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:23.492668 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-2069884c-5590-4948-8d85-a3e4e53c526c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2216c5cb9f184935960b2d3880aaf25f', 'request_id': 'req-2069884c-5590-4948-8d85-a3e4e53c526c'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:23.595544 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-2069884c-5590-4948-8d85-a3e4e53c526c admin admin] The token cache is being invalidate because project 2216c5cb9f184935960b2d3880aaf25f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:49:23.595544 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-2069884c-5590-4948-8d85-a3e4e53c526c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:23.595544 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-2069884c-5590-4948-8d85-a3e4e53c526c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:23.598009 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1363/2737] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:49:23 2022] DELETE /identity/v3/projects/2216c5cb9f184935960b2d3880aaf25f => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:23.598851 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3569931-245b-475c-8990-a257d9caed26 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:23.610166 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3569931-245b-475c-8990-a257d9caed26 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:23.669787 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5659c6c-3ddf-414b-85ca-bb5b2a2c35a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:23.670488 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5659c6c-3ddf-414b-85ca-bb5b2a2c35a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:23.670770 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5659c6c-3ddf-414b-85ca-bb5b2a2c35a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:23.671036 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5659c6c-3ddf-414b-85ca-bb5b2a2c35a3 admin admin] PATH_INFO: `/v3/users/81f7903718ca47aebf41179758763e01` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:23.672204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5659c6c-3ddf-414b-85ca-bb5b2a2c35a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81f7903718ca47aebf41179758763e01)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:23.674124 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5659c6c-3ddf-414b-85ca-bb5b2a2c35a3 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:23.905359 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c5659c6c-3ddf-414b-85ca-bb5b2a2c35a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81f7903718ca47aebf41179758763e01'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:23.908009 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c5659c6c-3ddf-414b-85ca-bb5b2a2c35a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81f7903718ca47aebf41179758763e01'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:23.908785 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c5659c6c-3ddf-414b-85ca-bb5b2a2c35a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81f7903718ca47aebf41179758763e01'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:23.922856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c5659c6c-3ddf-414b-85ca-bb5b2a2c35a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81f7903718ca47aebf41179758763e01'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:23.991246 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c5659c6c-3ddf-414b-85ca-bb5b2a2c35a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81f7903718ca47aebf41179758763e01'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:24.009054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:24.015212 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1375/2738] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:49:23 2022] DELETE /identity/v3/users/81f7903718ca47aebf41179758763e01 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:24.735320 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069884c-5590-4948-8d85-a3e4e53c526c admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:24.737203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2069884c-5590-4948-8d85-a3e4e53c526c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:24.751187 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069884c-5590-4948-8d85-a3e4e53c526c admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:24.841272 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde494d6-d1e4-4a2f-83e5-e3f937fcd796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:24.842076 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde494d6-d1e4-4a2f-83e5-e3f937fcd796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:24.842359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde494d6-d1e4-4a2f-83e5-e3f937fcd796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:24.843548 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde494d6-d1e4-4a2f-83e5-e3f937fcd796 admin admin] PATH_INFO: `/v3/projects/c2ebdf5cc5e54cec9d09cde288d0d273` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:24.844866 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde494d6-d1e4-4a2f-83e5-e3f937fcd796 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2ebdf5cc5e54cec9d09cde288d0d273)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:24.846951 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde494d6-d1e4-4a2f-83e5-e3f937fcd796 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:24.905341 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:25.003043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-dde494d6-d1e4-4a2f-83e5-e3f937fcd796 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2ebdf5cc5e54cec9d09cde288d0d273', 'request_id': 'req-dde494d6-d1e4-4a2f-83e5-e3f937fcd796'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:25.026527 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-dde494d6-d1e4-4a2f-83e5-e3f937fcd796 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2ebdf5cc5e54cec9d09cde288d0d273', 'request_id': 'req-dde494d6-d1e4-4a2f-83e5-e3f937fcd796'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:25.041473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-dde494d6-d1e4-4a2f-83e5-e3f937fcd796 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2ebdf5cc5e54cec9d09cde288d0d273', 'request_id': 'req-dde494d6-d1e4-4a2f-83e5-e3f937fcd796'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:25.121393 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-dde494d6-d1e4-4a2f-83e5-e3f937fcd796 admin admin] The token cache is being invalidate because project c2ebdf5cc5e54cec9d09cde288d0d273 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:49:25.121746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-dde494d6-d1e4-4a2f-83e5-e3f937fcd796 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:25.122880 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-dde494d6-d1e4-4a2f-83e5-e3f937fcd796 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:25.128552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1364/2739] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:49:24 2022] DELETE /identity/v3/projects/c2ebdf5cc5e54cec9d09cde288d0d273 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:25.129727 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5659c6c-3ddf-414b-85ca-bb5b2a2c35a3 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:25.132027 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c5659c6c-3ddf-414b-85ca-bb5b2a2c35a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:25.156454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5659c6c-3ddf-414b-85ca-bb5b2a2c35a3 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:25.230473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8575dd8-49e4-4b10-9257-de22b26c2151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:25.231263 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8575dd8-49e4-4b10-9257-de22b26c2151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:25.231550 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8575dd8-49e4-4b10-9257-de22b26c2151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:25.232424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8575dd8-49e4-4b10-9257-de22b26c2151 admin admin] PATH_INFO: `/v3/users/8d8b7e20cd4942eb8b834f79931c6981` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:25.234039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8575dd8-49e4-4b10-9257-de22b26c2151 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d8b7e20cd4942eb8b834f79931c6981)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:25.236193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8575dd8-49e4-4b10-9257-de22b26c2151 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:25.378532 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:25.416083 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f8575dd8-49e4-4b10-9257-de22b26c2151 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d8b7e20cd4942eb8b834f79931c6981'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:25.416836 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f8575dd8-49e4-4b10-9257-de22b26c2151 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d8b7e20cd4942eb8b834f79931c6981'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:25.417422 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f8575dd8-49e4-4b10-9257-de22b26c2151 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d8b7e20cd4942eb8b834f79931c6981'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:25.424811 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f8575dd8-49e4-4b10-9257-de22b26c2151 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d8b7e20cd4942eb8b834f79931c6981'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:25.529294 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f8575dd8-49e4-4b10-9257-de22b26c2151 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d8b7e20cd4942eb8b834f79931c6981'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:25.556603 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1376/2740] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:49:25 2022] DELETE /identity/v3/users/8d8b7e20cd4942eb8b834f79931c6981 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:49:26.181836 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde494d6-d1e4-4a2f-83e5-e3f937fcd796 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:26.183232 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-dde494d6-d1e4-4a2f-83e5-e3f937fcd796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:26.207293 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde494d6-d1e4-4a2f-83e5-e3f937fcd796 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:26.278677 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9afc02-0e73-4438-a09a-cc8fa2e38ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:26.279256 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9afc02-0e73-4438-a09a-cc8fa2e38ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:26.279531 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9afc02-0e73-4438-a09a-cc8fa2e38ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:26.279794 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9afc02-0e73-4438-a09a-cc8fa2e38ccc admin admin] PATH_INFO: `/v3/projects/d0865e5ed4d5448291ce53bbce1472b9` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:26.281054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9afc02-0e73-4438-a09a-cc8fa2e38ccc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0865e5ed4d5448291ce53bbce1472b9)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:26.283064 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9afc02-0e73-4438-a09a-cc8fa2e38ccc admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:26.420723 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ae9afc02-0e73-4438-a09a-cc8fa2e38ccc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0865e5ed4d5448291ce53bbce1472b9', 'request_id': 'req-ae9afc02-0e73-4438-a09a-cc8fa2e38ccc'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:26.428693 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ae9afc02-0e73-4438-a09a-cc8fa2e38ccc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0865e5ed4d5448291ce53bbce1472b9', 'request_id': 'req-ae9afc02-0e73-4438-a09a-cc8fa2e38ccc'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:26.434055 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ae9afc02-0e73-4438-a09a-cc8fa2e38ccc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0865e5ed4d5448291ce53bbce1472b9', 'request_id': 'req-ae9afc02-0e73-4438-a09a-cc8fa2e38ccc'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:26.528599 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ae9afc02-0e73-4438-a09a-cc8fa2e38ccc admin admin] The token cache is being invalidate because project d0865e5ed4d5448291ce53bbce1472b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:49:26.529240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ae9afc02-0e73-4438-a09a-cc8fa2e38ccc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:26.530001 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-ae9afc02-0e73-4438-a09a-cc8fa2e38ccc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:26.531659 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1365/2741] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:49:26 2022] DELETE /identity/v3/projects/d0865e5ed4d5448291ce53bbce1472b9 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:26.539320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5e411-c8bd-4397-8b97-c42095676362 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:26.539863 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5e411-c8bd-4397-8b97-c42095676362 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:26.540557 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5e411-c8bd-4397-8b97-c42095676362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:26.562753 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-a2c5e411-c8bd-4397-8b97-c42095676362 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:26.564340 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-a2c5e411-c8bd-4397-8b97-c42095676362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:26.633285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1377/2742] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:26.642042 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f284ab-d923-4a06-a519-567700a1e718 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:26.642374 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f284ab-d923-4a06-a519-567700a1e718 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:26.642703 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f284ab-d923-4a06-a519-567700a1e718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:26.668233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-f1f284ab-d923-4a06-a519-567700a1e718 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:26.669852 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f1f284ab-d923-4a06-a519-567700a1e718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:26.679330 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1366/2743] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:26.683334 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5e411-c8bd-4397-8b97-c42095676362 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:26.694386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5e411-c8bd-4397-8b97-c42095676362 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:26.699196 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a5096-9265-48d2-af90-c6e809858f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:26.699817 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a5096-9265-48d2-af90-c6e809858f7c admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:26.700092 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a5096-9265-48d2-af90-c6e809858f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:26.700358 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a5096-9265-48d2-af90-c6e809858f7c admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:26.701007 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a5096-9265-48d2-af90-c6e809858f7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:49:26.701342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a5096-9265-48d2-af90-c6e809858f7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:26.702940 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a5096-9265-48d2-af90-c6e809858f7c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:26.716664 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1378/2744] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:49:26.721373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f284ab-d923-4a06-a519-567700a1e718 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:26.732125 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f284ab-d923-4a06-a519-567700a1e718 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:26.737960 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc04fe-c17a-4384-8045-1e977362b94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:26.738787 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc04fe-c17a-4384-8045-1e977362b94b admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:26.739239 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc04fe-c17a-4384-8045-1e977362b94b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:26.739571 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc04fe-c17a-4384-8045-1e977362b94b admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:26.739843 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:26.740578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc04fe-c17a-4384-8045-1e977362b94b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:26.743552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc04fe-c17a-4384-8045-1e977362b94b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:26.867987 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1367/2745] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:49:26 2022] POST /identity/v3/projects => generated 379 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:49:26.872162 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a5096-9265-48d2-af90-c6e809858f7c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:26.882692 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a5096-9265-48d2-af90-c6e809858f7c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:26.983199 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b8574-f5f5-4405-b2ff-b104ecc654ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:26.984142 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b8574-f5f5-4405-b2ff-b104ecc654ab admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:26.984522 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b8574-f5f5-4405-b2ff-b104ecc654ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:26.984894 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b8574-f5f5-4405-b2ff-b104ecc654ab admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:26.985936 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b8574-f5f5-4405-b2ff-b104ecc654ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:26.988315 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b8574-f5f5-4405-b2ff-b104ecc654ab admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:27.085418 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1379/2746] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:49:26 2022] POST /identity/v3/users => generated 290 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:49:27.089319 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc04fe-c17a-4384-8045-1e977362b94b admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:27.101486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc04fe-c17a-4384-8045-1e977362b94b admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:27.106874 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc5d29-1975-4242-9c0b-bbd80219b91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:27.107659 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc5d29-1975-4242-9c0b-bbd80219b91c admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:27.108009 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc5d29-1975-4242-9c0b-bbd80219b91c admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:27.108352 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc5d29-1975-4242-9c0b-bbd80219b91c admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:27.109135 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc5d29-1975-4242-9c0b-bbd80219b91c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:27.111082 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc5d29-1975-4242-9c0b-bbd80219b91c admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:27.122923 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1368/2747] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:49:27.126708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b8574-f5f5-4405-b2ff-b104ecc654ab admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:27.149412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b8574-f5f5-4405-b2ff-b104ecc654ab admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:27.155004 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092194e7-6e58-4184-8065-5b75a2b40074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:27.155652 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092194e7-6e58-4184-8065-5b75a2b40074 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:27.155934 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092194e7-6e58-4184-8065-5b75a2b40074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:27.156207 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092194e7-6e58-4184-8065-5b75a2b40074 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:27.156889 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092194e7-6e58-4184-8065-5b75a2b40074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:27.158686 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092194e7-6e58-4184-8065-5b75a2b40074 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:27.170067 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1380/2748] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:49:27.174197 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc5d29-1975-4242-9c0b-bbd80219b91c admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:27.185479 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc5d29-1975-4242-9c0b-bbd80219b91c admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:27.190547 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c74e0-ee1e-4e6b-ad58-f2fe7acbd2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:27.191479 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c74e0-ee1e-4e6b-ad58-f2fe7acbd2e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:27.191954 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c74e0-ee1e-4e6b-ad58-f2fe7acbd2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:27.192419 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c74e0-ee1e-4e6b-ad58-f2fe7acbd2e5 admin admin] PATH_INFO: `/v3/projects/ab0a5a54971649e2ba18fa3e46ff7316/users/3c1d463ccd954ff79b052bc3a215f466/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:27.220486 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c74e0-ee1e-4e6b-ad58-f2fe7acbd2e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab0a5a54971649e2ba18fa3e46ff7316, user_id=3c1d463ccd954ff79b052bc3a215f466, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:27.223362 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c74e0-ee1e-4e6b-ad58-f2fe7acbd2e5 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:27.323983 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1369/2749] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:49:27 2022] PUT /identity/v3/projects/ab0a5a54971649e2ba18fa3e46ff7316/users/3c1d463ccd954ff79b052bc3a215f466/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:27.336260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650be3de-aacd-4f65-8ce2-778975a459ec None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:27.336471 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650be3de-aacd-4f65-8ce2-778975a459ec None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:27.336754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650be3de-aacd-4f65-8ce2-778975a459ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:27.370121 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-650be3de-aacd-4f65-8ce2-778975a459ec None None] MFA Rules not processed for user `3c1d463ccd954ff79b052bc3a215f466`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:27.371341 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-650be3de-aacd-4f65-8ce2-778975a459ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:27.453661 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1381/2750] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:27.457868 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c74e0-ee1e-4e6b-ad58-f2fe7acbd2e5 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:27.462650 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:27.526994 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c74e0-ee1e-4e6b-ad58-f2fe7acbd2e5 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:27.531889 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aacec43-307b-4f09-a911-842c92783c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:27.532534 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacec43-307b-4f09-a911-842c92783c35 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:27.532810 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacec43-307b-4f09-a911-842c92783c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:27.533306 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacec43-307b-4f09-a911-842c92783c35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:27.533936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aacec43-307b-4f09-a911-842c92783c35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:27.535968 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aacec43-307b-4f09-a911-842c92783c35 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:27.654287 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1370/2751] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:49:27 2022] POST /identity/v3/projects => generated 381 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:49:27.658367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650be3de-aacd-4f65-8ce2-778975a459ec None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:27.673113 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650be3de-aacd-4f65-8ce2-778975a459ec None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:27.746383 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fea38-78aa-4e02-8f16-0cf3fdd45f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:27.747261 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fea38-78aa-4e02-8f16-0cf3fdd45f12 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:27.747642 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fea38-78aa-4e02-8f16-0cf3fdd45f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:27.748006 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fea38-78aa-4e02-8f16-0cf3fdd45f12 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:27.748738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3fea38-78aa-4e02-8f16-0cf3fdd45f12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:27.750888 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3fea38-78aa-4e02-8f16-0cf3fdd45f12 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:27.815045 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:27.861339 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1382/2752] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:49:27 2022] POST /identity/v3/users => generated 297 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:49:27.861545 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aacec43-307b-4f09-a911-842c92783c35 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:27.864399 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aacec43-307b-4f09-a911-842c92783c35 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:27.869382 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9037b1-314b-4239-a85a-0a799a03d558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:27.869990 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9037b1-314b-4239-a85a-0a799a03d558 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:27.870268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9037b1-314b-4239-a85a-0a799a03d558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:27.870544 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9037b1-314b-4239-a85a-0a799a03d558 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:27.871216 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9037b1-314b-4239-a85a-0a799a03d558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:27.872869 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9037b1-314b-4239-a85a-0a799a03d558 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:27.885933 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1371/2753] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:49:27.889869 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fea38-78aa-4e02-8f16-0cf3fdd45f12 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:27.900734 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fea38-78aa-4e02-8f16-0cf3fdd45f12 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:27.905484 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524bdff3-0101-417a-b407-49bb7961f04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:27.906260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524bdff3-0101-417a-b407-49bb7961f04e admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:27.906631 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524bdff3-0101-417a-b407-49bb7961f04e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:27.906977 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524bdff3-0101-417a-b407-49bb7961f04e admin admin] PATH_INFO: `/v3/projects/38e44af470bf46a1825fba59e8feaa7b/users/f1b827a7c1564cada85d8c593e6e18dc/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:27.923634 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524bdff3-0101-417a-b407-49bb7961f04e admin admin] RBAC: Authorizing `identity:create_grant(project_id=38e44af470bf46a1825fba59e8feaa7b, user_id=f1b827a7c1564cada85d8c593e6e18dc, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:27.926421 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524bdff3-0101-417a-b407-49bb7961f04e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:28.050025 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1383/2754] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:49:27 2022] PUT /identity/v3/projects/38e44af470bf46a1825fba59e8feaa7b/users/f1b827a7c1564cada85d8c593e6e18dc/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:49:28.054436 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9037b1-314b-4239-a85a-0a799a03d558 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:28.064986 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9037b1-314b-4239-a85a-0a799a03d558 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:28.124849 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcab174-74c5-4aff-aa6e-b23de27cd6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:28.125561 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcab174-74c5-4aff-aa6e-b23de27cd6ab admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:28.125834 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcab174-74c5-4aff-aa6e-b23de27cd6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:28.126101 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcab174-74c5-4aff-aa6e-b23de27cd6ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:28.126732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcab174-74c5-4aff-aa6e-b23de27cd6ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:28.128293 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcab174-74c5-4aff-aa6e-b23de27cd6ab admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:28.141503 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1372/2755] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:49:28.144826 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524bdff3-0101-417a-b407-49bb7961f04e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:28.157039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524bdff3-0101-417a-b407-49bb7961f04e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:28.163133 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca122d-afca-4787-9539-f52dddf4807a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:28.163856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca122d-afca-4787-9539-f52dddf4807a admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:28.164236 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca122d-afca-4787-9539-f52dddf4807a admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:28.164516 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca122d-afca-4787-9539-f52dddf4807a admin admin] PATH_INFO: `/v3/projects/38e44af470bf46a1825fba59e8feaa7b/users/f1b827a7c1564cada85d8c593e6e18dc/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:28.167357 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ca122d-afca-4787-9539-f52dddf4807a admin admin] RBAC: Authorizing `identity:create_grant(project_id=38e44af470bf46a1825fba59e8feaa7b, user_id=f1b827a7c1564cada85d8c593e6e18dc, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:28.171004 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ca122d-afca-4787-9539-f52dddf4807a admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:28.189376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1384/2756] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:49:28 2022] PUT /identity/v3/projects/38e44af470bf46a1825fba59e8feaa7b/users/f1b827a7c1564cada85d8c593e6e18dc/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:28.198720 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020e0c8-0adc-4130-a681-0fdf513b2cbe None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:28.199251 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020e0c8-0adc-4130-a681-0fdf513b2cbe None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:28.199424 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020e0c8-0adc-4130-a681-0fdf513b2cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:28.233353 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-6020e0c8-0adc-4130-a681-0fdf513b2cbe None None] MFA Rules not processed for user `f1b827a7c1564cada85d8c593e6e18dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:28.234834 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6020e0c8-0adc-4130-a681-0fdf513b2cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:28.361628 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1373/2757] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:28 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:49:28.369685 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca122d-afca-4787-9539-f52dddf4807a admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:28.371431 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d3ca122d-afca-4787-9539-f52dddf4807a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:28.420172 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:28.468529 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca122d-afca-4787-9539-f52dddf4807a admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:28.473704 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00da49-b9de-408f-aaa0-971b4c32728b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:28.474326 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00da49-b9de-408f-aaa0-971b4c32728b service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:28.474764 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00da49-b9de-408f-aaa0-971b4c32728b service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:28.475198 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00da49-b9de-408f-aaa0-971b4c32728b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:28.487265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f00da49-b9de-408f-aaa0-971b4c32728b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:28.488705 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f00da49-b9de-408f-aaa0-971b4c32728b service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:28.502638 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1385/2758] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:28 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:49:28.657875 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020e0c8-0adc-4130-a681-0fdf513b2cbe None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:28.668045 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020e0c8-0adc-4130-a681-0fdf513b2cbe None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:28.671993 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b4f9d-821d-413f-8a03-e8afd73e1ec6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:28.675531 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b4f9d-821d-413f-8a03-e8afd73e1ec6 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:28.675531 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b4f9d-821d-413f-8a03-e8afd73e1ec6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:28.675531 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b4f9d-821d-413f-8a03-e8afd73e1ec6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:28.756511 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b4f9d-821d-413f-8a03-e8afd73e1ec6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:28.757341 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b4f9d-821d-413f-8a03-e8afd73e1ec6 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:28.770237 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1374/2759] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:49:29.385405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00da49-b9de-408f-aaa0-971b4c32728b service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:29.464606 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f22ff-987f-4585-93f8-935f2a7cb9f7 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:29.465514 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f22ff-987f-4585-93f8-935f2a7cb9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:29.474331 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f22ff-987f-4585-93f8-935f2a7cb9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:29.504566 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-eb8f22ff-987f-4585-93f8-935f2a7cb9f7 None None] MFA Rules not processed for user `891658773e7546c78c3213a3f5233fd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:29.505548 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-eb8f22ff-987f-4585-93f8-935f2a7cb9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:29.544742 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00da49-b9de-408f-aaa0-971b4c32728b service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:29.561816 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb9307-b56a-45f6-9f39-3ed0b2ce3664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:29.562595 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb9307-b56a-45f6-9f39-3ed0b2ce3664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:29.562875 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb9307-b56a-45f6-9f39-3ed0b2ce3664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:29.563315 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb9307-b56a-45f6-9f39-3ed0b2ce3664 admin admin] PATH_INFO: `/v3/users/3c1d463ccd954ff79b052bc3a215f466` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:29.564677 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb9307-b56a-45f6-9f39-3ed0b2ce3664 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c1d463ccd954ff79b052bc3a215f466)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:29.566828 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb9307-b56a-45f6-9f39-3ed0b2ce3664 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:29.635992 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1375/2760] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:49:29.639484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f22ff-987f-4585-93f8-935f2a7cb9f7 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:29.652598 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-eb8f22ff-987f-4585-93f8-935f2a7cb9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:29.704387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f22ff-987f-4585-93f8-935f2a7cb9f7 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:29.711691 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59faa78-8d82-477f-8cc7-8fe26f3d0307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:29.711691 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59faa78-8d82-477f-8cc7-8fe26f3d0307 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:29.712001 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59faa78-8d82-477f-8cc7-8fe26f3d0307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:29.712263 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59faa78-8d82-477f-8cc7-8fe26f3d0307 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:29.712870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59faa78-8d82-477f-8cc7-8fe26f3d0307 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:29.715770 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59faa78-8d82-477f-8cc7-8fe26f3d0307 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:29.845518 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1376/2761] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:49:29 2022] POST /identity/v3/projects => generated 383 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:49:29.849273 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59faa78-8d82-477f-8cc7-8fe26f3d0307 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:29.854606 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9fdb9307-b56a-45f6-9f39-3ed0b2ce3664 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c1d463ccd954ff79b052bc3a215f466'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:29.855851 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9fdb9307-b56a-45f6-9f39-3ed0b2ce3664 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c1d463ccd954ff79b052bc3a215f466'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:29.857078 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9fdb9307-b56a-45f6-9f39-3ed0b2ce3664 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c1d463ccd954ff79b052bc3a215f466'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:29.868533 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9fdb9307-b56a-45f6-9f39-3ed0b2ce3664 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c1d463ccd954ff79b052bc3a215f466'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:29.921578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59faa78-8d82-477f-8cc7-8fe26f3d0307 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:29.926427 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f3ed1-f498-4e8a-b24d-ad62486ae3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:29.927194 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f3ed1-f498-4e8a-b24d-ad62486ae3e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:29.928054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f3ed1-f498-4e8a-b24d-ad62486ae3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:29.928386 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f3ed1-f498-4e8a-b24d-ad62486ae3e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:29.929103 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f3ed1-f498-4e8a-b24d-ad62486ae3e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:29.931097 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f3ed1-f498-4e8a-b24d-ad62486ae3e4 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:29.941939 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:29.961949 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-9fdb9307-b56a-45f6-9f39-3ed0b2ce3664 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c1d463ccd954ff79b052bc3a215f466'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:29.963395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:29.981243 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1386/2762] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:49:29 2022] DELETE /identity/v3/users/3c1d463ccd954ff79b052bc3a215f466 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:29.985751 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b35b0-2b70-48fe-8ec4-c317ca942b3f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:29.986039 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b35b0-2b70-48fe-8ec4-c317ca942b3f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:29.986314 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b35b0-2b70-48fe-8ec4-c317ca942b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:30.004583 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-260b35b0-2b70-48fe-8ec4-c317ca942b3f None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:30.005879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-260b35b0-2b70-48fe-8ec4-c317ca942b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:30.055619 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1377/2763] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:49:29 2022] POST /identity/v3/users => generated 298 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:49:30.059450 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f3ed1-f498-4e8a-b24d-ad62486ae3e4 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:30.061085 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f83f3ed1-f498-4e8a-b24d-ad62486ae3e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:30.067359 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1387/2764] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:49:30.077598 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b35b0-2b70-48fe-8ec4-c317ca942b3f None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:30.078988 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-260b35b0-2b70-48fe-8ec4-c317ca942b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:30.081425 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f3ed1-f498-4e8a-b24d-ad62486ae3e4 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:30.087259 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce91f4-125e-4a32-9f56-c71af6e4e79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:30.087890 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce91f4-125e-4a32-9f56-c71af6e4e79c admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:30.088180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce91f4-125e-4a32-9f56-c71af6e4e79c admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:30.088505 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce91f4-125e-4a32-9f56-c71af6e4e79c admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:30.089183 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce91f4-125e-4a32-9f56-c71af6e4e79c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:30.091212 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce91f4-125e-4a32-9f56-c71af6e4e79c admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:30.108060 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1378/2765] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:49:30.109425 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce91f4-125e-4a32-9f56-c71af6e4e79c admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:30.119191 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce91f4-125e-4a32-9f56-c71af6e4e79c admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:30.123818 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b60ab-eaeb-4738-b8bd-0a074cda5fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:30.124456 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b60ab-eaeb-4738-b8bd-0a074cda5fee admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:30.124743 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b60ab-eaeb-4738-b8bd-0a074cda5fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:30.125001 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b60ab-eaeb-4738-b8bd-0a074cda5fee admin admin] PATH_INFO: `/v3/projects/75dc214097cc42d1b882d5bfcd4d7654/users/54c33c5b6a6e4741a99670befefff1f1/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:30.141531 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b60ab-eaeb-4738-b8bd-0a074cda5fee admin admin] RBAC: Authorizing `identity:create_grant(project_id=75dc214097cc42d1b882d5bfcd4d7654, user_id=54c33c5b6a6e4741a99670befefff1f1, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:30.142184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b35b0-2b70-48fe-8ec4-c317ca942b3f None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:30.144305 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b60ab-eaeb-4738-b8bd-0a074cda5fee admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:30.146094 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f099cc8-8914-46ff-846e-3d1172351196 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:30.147390 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f099cc8-8914-46ff-846e-3d1172351196 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:30.147667 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f099cc8-8914-46ff-846e-3d1172351196 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:30.147943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f099cc8-8914-46ff-846e-3d1172351196 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:30.158183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f099cc8-8914-46ff-846e-3d1172351196 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:30.159624 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f099cc8-8914-46ff-846e-3d1172351196 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:30.169767 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1388/2766] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:49:30.214403 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1379/2767] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:49:30 2022] PUT /identity/v3/projects/75dc214097cc42d1b882d5bfcd4d7654/users/54c33c5b6a6e4741a99670befefff1f1/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:30.218293 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f099cc8-8914-46ff-846e-3d1172351196 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:30.228042 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f099cc8-8914-46ff-846e-3d1172351196 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:30.285663 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abeb4b0-cd23-46a7-84cb-09153225b685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:30.286269 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abeb4b0-cd23-46a7-84cb-09153225b685 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:30.286545 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abeb4b0-cd23-46a7-84cb-09153225b685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:30.286832 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abeb4b0-cd23-46a7-84cb-09153225b685 admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:30.287959 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abeb4b0-cd23-46a7-84cb-09153225b685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:30.289610 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abeb4b0-cd23-46a7-84cb-09153225b685 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:30.299573 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1389/2768] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:49:30.304427 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b60ab-eaeb-4738-b8bd-0a074cda5fee admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:30.315351 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b60ab-eaeb-4738-b8bd-0a074cda5fee admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:30.320074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9137e-c67d-46be-9ee2-3d1b2a1e79bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:30.320734 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9137e-c67d-46be-9ee2-3d1b2a1e79bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:30.321010 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9137e-c67d-46be-9ee2-3d1b2a1e79bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:30.321359 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9137e-c67d-46be-9ee2-3d1b2a1e79bd admin admin] PATH_INFO: `/v3/projects/75dc214097cc42d1b882d5bfcd4d7654/users/54c33c5b6a6e4741a99670befefff1f1/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:30.323243 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac9137e-c67d-46be-9ee2-3d1b2a1e79bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=75dc214097cc42d1b882d5bfcd4d7654, user_id=54c33c5b6a6e4741a99670befefff1f1, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:30.325941 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac9137e-c67d-46be-9ee2-3d1b2a1e79bd admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:30.336832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1380/2769] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:49:30 2022] PUT /identity/v3/projects/75dc214097cc42d1b882d5bfcd4d7654/users/54c33c5b6a6e4741a99670befefff1f1/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:49:30.632470 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abeb4b0-cd23-46a7-84cb-09153225b685 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:30.634288 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4abeb4b0-cd23-46a7-84cb-09153225b685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:30.647779 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abeb4b0-cd23-46a7-84cb-09153225b685 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:30.734464 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac0c04-be75-4586-9965-193605f2e817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:30.735363 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac0c04-be75-4586-9965-193605f2e817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:30.735844 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac0c04-be75-4586-9965-193605f2e817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:30.736210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac0c04-be75-4586-9965-193605f2e817 admin admin] PATH_INFO: `/v3/projects/ab0a5a54971649e2ba18fa3e46ff7316` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:30.739405 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac0c04-be75-4586-9965-193605f2e817 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab0a5a54971649e2ba18fa3e46ff7316)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:30.742590 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac0c04-be75-4586-9965-193605f2e817 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:30.768988 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9137e-c67d-46be-9ee2-3d1b2a1e79bd admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:30.772499 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-4ac9137e-c67d-46be-9ee2-3d1b2a1e79bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:30.853526 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9137e-c67d-46be-9ee2-3d1b2a1e79bd admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:30.858886 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e37034-f2ee-4790-8b3e-47776c6aa1b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:30.859732 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e37034-f2ee-4790-8b3e-47776c6aa1b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:30.860225 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e37034-f2ee-4790-8b3e-47776c6aa1b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:30.860569 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e37034-f2ee-4790-8b3e-47776c6aa1b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:30.865521 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-76e37034-f2ee-4790-8b3e-47776c6aa1b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:30.900266 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fcac0c04-be75-4586-9965-193605f2e817 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab0a5a54971649e2ba18fa3e46ff7316', 'request_id': 'req-fcac0c04-be75-4586-9965-193605f2e817'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:30.954389 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e37034-f2ee-4790-8b3e-47776c6aa1b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:30.956957 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e37034-f2ee-4790-8b3e-47776c6aa1b6 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:30.971064 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1381/2770] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:49:30.979688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fcac0c04-be75-4586-9965-193605f2e817 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab0a5a54971649e2ba18fa3e46ff7316', 'request_id': 'req-fcac0c04-be75-4586-9965-193605f2e817'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:30.986975 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fcac0c04-be75-4586-9965-193605f2e817 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab0a5a54971649e2ba18fa3e46ff7316', 'request_id': 'req-fcac0c04-be75-4586-9965-193605f2e817'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:31.000952 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fcac0c04-be75-4586-9965-193605f2e817 admin admin] The token cache is being invalidate because project ab0a5a54971649e2ba18fa3e46ff7316 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:49:31.001298 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fcac0c04-be75-4586-9965-193605f2e817 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:31.002083 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-fcac0c04-be75-4586-9965-193605f2e817 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:31.004737 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1390/2771] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:49:30 2022] DELETE /identity/v3/projects/ab0a5a54971649e2ba18fa3e46ff7316 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:31.015877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e37034-f2ee-4790-8b3e-47776c6aa1b6 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:31.022803 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e37034-f2ee-4790-8b3e-47776c6aa1b6 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:31.099302 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc1face-63f6-4e6c-be7f-e1e42c2f1297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:31.099302 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1face-63f6-4e6c-be7f-e1e42c2f1297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:31.099302 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1face-63f6-4e6c-be7f-e1e42c2f1297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:31.099302 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1face-63f6-4e6c-be7f-e1e42c2f1297 admin admin] PATH_INFO: `/v3/users/f1b827a7c1564cada85d8c593e6e18dc` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:31.101960 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc1face-63f6-4e6c-be7f-e1e42c2f1297 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1b827a7c1564cada85d8c593e6e18dc)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:31.104985 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc1face-63f6-4e6c-be7f-e1e42c2f1297 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:31.106044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:31.388175 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-acc1face-63f6-4e6c-be7f-e1e42c2f1297 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1b827a7c1564cada85d8c593e6e18dc'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:31.389578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-acc1face-63f6-4e6c-be7f-e1e42c2f1297 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1b827a7c1564cada85d8c593e6e18dc'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:31.390330 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-acc1face-63f6-4e6c-be7f-e1e42c2f1297 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1b827a7c1564cada85d8c593e6e18dc'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:31.399609 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-acc1face-63f6-4e6c-be7f-e1e42c2f1297 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1b827a7c1564cada85d8c593e6e18dc'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:31.563669 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-acc1face-63f6-4e6c-be7f-e1e42c2f1297 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1b827a7c1564cada85d8c593e6e18dc'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:31.656906 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1382/2772] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:49:31 2022] DELETE /identity/v3/users/f1b827a7c1564cada85d8c593e6e18dc => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:49:32.245440 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac0c04-be75-4586-9965-193605f2e817 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:32.247989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fcac0c04-be75-4586-9965-193605f2e817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:32.274382 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac0c04-be75-4586-9965-193605f2e817 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:32.362059 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a6a76-18b1-4a07-81e6-80d52d82efc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:32.362976 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a6a76-18b1-4a07-81e6-80d52d82efc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:32.363326 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a6a76-18b1-4a07-81e6-80d52d82efc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:32.363818 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a6a76-18b1-4a07-81e6-80d52d82efc9 admin admin] PATH_INFO: `/v3/projects/38e44af470bf46a1825fba59e8feaa7b` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:32.365122 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a6a76-18b1-4a07-81e6-80d52d82efc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38e44af470bf46a1825fba59e8feaa7b)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:32.369414 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a6a76-18b1-4a07-81e6-80d52d82efc9 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:32.389097 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:32.508174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c14a6a76-18b1-4a07-81e6-80d52d82efc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38e44af470bf46a1825fba59e8feaa7b', 'request_id': 'req-c14a6a76-18b1-4a07-81e6-80d52d82efc9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:32.563888 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c14a6a76-18b1-4a07-81e6-80d52d82efc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38e44af470bf46a1825fba59e8feaa7b', 'request_id': 'req-c14a6a76-18b1-4a07-81e6-80d52d82efc9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:32.574543 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c14a6a76-18b1-4a07-81e6-80d52d82efc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38e44af470bf46a1825fba59e8feaa7b', 'request_id': 'req-c14a6a76-18b1-4a07-81e6-80d52d82efc9'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:32.587856 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c14a6a76-18b1-4a07-81e6-80d52d82efc9 admin admin] The token cache is being invalidate because project 38e44af470bf46a1825fba59e8feaa7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:49:32.588407 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c14a6a76-18b1-4a07-81e6-80d52d82efc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:32.589094 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c14a6a76-18b1-4a07-81e6-80d52d82efc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:49:32.590819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1391/2773] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:49:32 2022] DELETE /identity/v3/projects/38e44af470bf46a1825fba59e8feaa7b => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:32.595771 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd7a1e-bd4b-42f1-910b-59bce48e051e None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:32.596444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd7a1e-bd4b-42f1-910b-59bce48e051e None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:32.596444 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd7a1e-bd4b-42f1-910b-59bce48e051e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:32.619274 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-87dd7a1e-bd4b-42f1-910b-59bce48e051e None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:32.620556 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-87dd7a1e-bd4b-42f1-910b-59bce48e051e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:32.686440 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1383/2774] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:49:32.695468 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0644fb7-d69b-4b68-8682-71822bad9259 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:32.696001 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0644fb7-d69b-4b68-8682-71822bad9259 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:32.696484 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0644fb7-d69b-4b68-8682-71822bad9259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:32.726879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-b0644fb7-d69b-4b68-8682-71822bad9259 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:32.728360 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b0644fb7-d69b-4b68-8682-71822bad9259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:32.736702 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1392/2775] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:32.741312 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd7a1e-bd4b-42f1-910b-59bce48e051e None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:32.757683 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd7a1e-bd4b-42f1-910b-59bce48e051e None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:32.762817 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9bdfe-710c-4d16-b430-319ae07c0f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:32.764358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9bdfe-710c-4d16-b430-319ae07c0f94 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:32.764759 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9bdfe-710c-4d16-b430-319ae07c0f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:32.765169 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9bdfe-710c-4d16-b430-319ae07c0f94 admin admin] PATH_INFO: `/v3/domains` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:32.766511 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9bdfe-710c-4d16-b430-319ae07c0f94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80262) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:49:32.768177 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9bdfe-710c-4d16-b430-319ae07c0f94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:32.770201 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9bdfe-710c-4d16-b430-319ae07c0f94 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:32.787984 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1384/2776] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:49:32.789256 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0644fb7-d69b-4b68-8682-71822bad9259 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:32.798773 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0644fb7-d69b-4b68-8682-71822bad9259 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:32.803493 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7b1d4-f166-4ed4-bf96-861e330e6c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:32.804093 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7b1d4-f166-4ed4-bf96-861e330e6c4c admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:32.804367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7b1d4-f166-4ed4-bf96-861e330e6c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:32.804632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7b1d4-f166-4ed4-bf96-861e330e6c4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:32.805293 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7b1d4-f166-4ed4-bf96-861e330e6c4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:32.807220 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7b1d4-f166-4ed4-bf96-861e330e6c4c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:32.933634 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1393/2777] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:49:32 2022] POST /identity/v3/projects => generated 375 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:49:32.937961 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9bdfe-710c-4d16-b430-319ae07c0f94 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:32.951753 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9bdfe-710c-4d16-b430-319ae07c0f94 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:33.028620 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:33.072194 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d2340-e755-45ee-a25b-46e2bb0d4670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:33.073031 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d2340-e755-45ee-a25b-46e2bb0d4670 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:33.073991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d2340-e755-45ee-a25b-46e2bb0d4670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:33.074670 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d2340-e755-45ee-a25b-46e2bb0d4670 admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:33.075524 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99d2340-e755-45ee-a25b-46e2bb0d4670 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:33.080749 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99d2340-e755-45ee-a25b-46e2bb0d4670 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:33.233015 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1385/2778] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:49:32 2022] POST /identity/v3/users => generated 288 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:49:33.237877 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7b1d4-f166-4ed4-bf96-861e330e6c4c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:33.306113 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7b1d4-f166-4ed4-bf96-861e330e6c4c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:33.311754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e5b66-d5f4-4640-b2e7-8961c85ca55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:33.312653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e5b66-d5f4-4640-b2e7-8961c85ca55c admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:33.313135 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e5b66-d5f4-4640-b2e7-8961c85ca55c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:33.313654 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e5b66-d5f4-4640-b2e7-8961c85ca55c admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:33.315516 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e5b66-d5f4-4640-b2e7-8961c85ca55c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:33.318270 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e5b66-d5f4-4640-b2e7-8961c85ca55c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:33.332753 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1394/2779] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:49:33.337480 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d2340-e755-45ee-a25b-46e2bb0d4670 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:33.350402 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d2340-e755-45ee-a25b-46e2bb0d4670 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:33.357482 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3167749b-5b4e-4d96-8506-58d182956722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:33.358388 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167749b-5b4e-4d96-8506-58d182956722 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:33.358741 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167749b-5b4e-4d96-8506-58d182956722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:33.359079 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167749b-5b4e-4d96-8506-58d182956722 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:33.360508 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3167749b-5b4e-4d96-8506-58d182956722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:33.362499 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3167749b-5b4e-4d96-8506-58d182956722 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:33.382020 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1386/2780] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:49:33.383666 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e5b66-d5f4-4640-b2e7-8961c85ca55c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:33.393911 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e5b66-d5f4-4640-b2e7-8961c85ca55c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:33.397454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:33.401031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd2229-ba0c-4286-8a2e-f9c621eae398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:33.401774 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd2229-ba0c-4286-8a2e-f9c621eae398 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:33.406371 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd2229-ba0c-4286-8a2e-f9c621eae398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:33.406371 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd2229-ba0c-4286-8a2e-f9c621eae398 admin admin] PATH_INFO: `/v3/projects/6a4e7ea0347a480b971c4253f2e4b436/users/b603187c9f7346278323ed07a75828ac/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:33.442070 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd2229-ba0c-4286-8a2e-f9c621eae398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a4e7ea0347a480b971c4253f2e4b436, user_id=b603187c9f7346278323ed07a75828ac, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:33.445082 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd2229-ba0c-4286-8a2e-f9c621eae398 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:33.553490 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1395/2781] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:49:33 2022] PUT /identity/v3/projects/6a4e7ea0347a480b971c4253f2e4b436/users/b603187c9f7346278323ed07a75828ac/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:49:33.629805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09a276-0be8-4300-9314-d605307ec9f8 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:33.630292 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09a276-0be8-4300-9314-d605307ec9f8 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:33.630433 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09a276-0be8-4300-9314-d605307ec9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:33.715544 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-8a09a276-0be8-4300-9314-d605307ec9f8 None None] MFA Rules not processed for user `b603187c9f7346278323ed07a75828ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:33.716793 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8a09a276-0be8-4300-9314-d605307ec9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:33.808805 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1387/2782] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:33.822161 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd2229-ba0c-4286-8a2e-f9c621eae398 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:33.824003 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-57bd2229-ba0c-4286-8a2e-f9c621eae398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:33.979997 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd2229-ba0c-4286-8a2e-f9c621eae398 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:33.984147 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0470a-a838-4876-90e3-900b8e54ca28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:33.984843 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0470a-a838-4876-90e3-900b8e54ca28 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:33.989816 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0470a-a838-4876-90e3-900b8e54ca28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:33.989816 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0470a-a838-4876-90e3-900b8e54ca28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:33.998082 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0470a-a838-4876-90e3-900b8e54ca28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:34.000052 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0470a-a838-4876-90e3-900b8e54ca28 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:34.015213 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1396/2783] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:49:36.795087 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0862fef-be56-40b3-b693-8673a579899b None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:36.795648 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0862fef-be56-40b3-b693-8673a579899b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:36.795648 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0862fef-be56-40b3-b693-8673a579899b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:36.827789 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-a0862fef-be56-40b3-b693-8673a579899b None None] MFA Rules not processed for user `54c33c5b6a6e4741a99670befefff1f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:36.829016 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a0862fef-be56-40b3-b693-8673a579899b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:36.901179 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1388/2784] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:36 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:36.912959 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0470a-a838-4876-90e3-900b8e54ca28 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:36.914531 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-86c0470a-a838-4876-90e3-900b8e54ca28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:36.939187 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:36.994847 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0470a-a838-4876-90e3-900b8e54ca28 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:36.994847 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415b000-24f8-4f9f-821c-f2fb171763d2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '627171bf2a2e4a21ba6f800879f8c665', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:36.994847 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415b000-24f8-4f9f-821c-f2fb171763d2 service glance] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:36.994847 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415b000-24f8-4f9f-821c-f2fb171763d2 service glance] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:36.995537 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415b000-24f8-4f9f-821c-f2fb171763d2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:36.995537 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6415b000-24f8-4f9f-821c-f2fb171763d2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:37.052629 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6415b000-24f8-4f9f-821c-f2fb171763d2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:37.059382 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6415b000-24f8-4f9f-821c-f2fb171763d2 service glance] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:37.070983 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1397/2785] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:49:37.315484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77f996-09cd-4bb8-b979-906bd1b4dc4b None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:37.315767 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77f996-09cd-4bb8-b979-906bd1b4dc4b None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:37.316033 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77f996-09cd-4bb8-b979-906bd1b4dc4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:37.328930 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:37.345492 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-9a77f996-09cd-4bb8-b979-906bd1b4dc4b None None] MFA Rules not processed for user `7f3422c6f9a041ceba851b71bd15d0df`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:37.346668 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-9a77f996-09cd-4bb8-b979-906bd1b4dc4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:37.413175 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1389/2786] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 08:49:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:37.421734 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415b000-24f8-4f9f-821c-f2fb171763d2 service glance] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:37.422917 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-6415b000-24f8-4f9f-821c-f2fb171763d2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:37.498104 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415b000-24f8-4f9f-821c-f2fb171763d2 service glance] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:37.503943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c519dab-02a2-47f4-b737-ffb565906ad3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e69c0e02ad944f2a85fc5318018e5cb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:37.504681 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c519dab-02a2-47f4-b737-ffb565906ad3 service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:37.505056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c519dab-02a2-47f4-b737-ffb565906ad3 service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:37.506222 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c519dab-02a2-47f4-b737-ffb565906ad3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:37.516467 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c519dab-02a2-47f4-b737-ffb565906ad3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:37.518217 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c519dab-02a2-47f4-b737-ffb565906ad3 service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:37.527772 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1398/2787] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 08:49:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 08:49:45.986172 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed10856-ede8-4d00-a475-2b070095c372 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:45.987023 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed10856-ede8-4d00-a475-2b070095c372 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:45.987448 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed10856-ede8-4d00-a475-2b070095c372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:46.022205 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-eed10856-ede8-4d00-a475-2b070095c372 None None] MFA Rules not processed for user `cd870619247a47ac810fea1aa636ac65`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:46.023811 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-eed10856-ede8-4d00-a475-2b070095c372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:46.113661 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1390/2788] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:47.523989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=71 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:48.645335 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cca2f-b2a5-49f2-9682-5aae7da9b52b None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:48.646449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cca2f-b2a5-49f2-9682-5aae7da9b52b None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:48.646449 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cca2f-b2a5-49f2-9682-5aae7da9b52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:48.664922 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-518cca2f-b2a5-49f2-9682-5aae7da9b52b None None] MFA Rules not processed for user `891658773e7546c78c3213a3f5233fd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:48.666147 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-518cca2f-b2a5-49f2-9682-5aae7da9b52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:48.673082 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1399/2789] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:49:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:49.369294 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed10856-ede8-4d00-a475-2b070095c372 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:49.371048 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-eed10856-ede8-4d00-a475-2b070095c372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:49.514730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed10856-ede8-4d00-a475-2b070095c372 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:49.521410 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2602f49d-1bf2-4aaf-a2a2-2debf3582abe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c23a782b9b4144bace1248c3b2e754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:49.522162 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602f49d-1bf2-4aaf-a2a2-2debf3582abe service placement] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:49.522454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602f49d-1bf2-4aaf-a2a2-2debf3582abe service placement] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:49.522705 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602f49d-1bf2-4aaf-a2a2-2debf3582abe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:49.524543 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2602f49d-1bf2-4aaf-a2a2-2debf3582abe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:49.538364 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2602f49d-1bf2-4aaf-a2a2-2debf3582abe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:49.540035 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2602f49d-1bf2-4aaf-a2a2-2debf3582abe service placement] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:49.552227 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1391/2790] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:49:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:49:58.659578 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:58.867168 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fe35d-f9c4-4312-9957-e7083a364066 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:58.867545 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fe35d-f9c4-4312-9957-e7083a364066 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:58.867881 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fe35d-f9c4-4312-9957-e7083a364066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:58.887152 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-185fe35d-f9c4-4312-9957-e7083a364066 None None] MFA Rules not processed for user `7f3422c6f9a041ceba851b71bd15d0df`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:49:58.888353 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-185fe35d-f9c4-4312-9957-e7083a364066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:49:58.895591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1400/2791] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 08:49:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:49:58.912086 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2602f49d-1bf2-4aaf-a2a2-2debf3582abe service placement] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:49:58.923642 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2602f49d-1bf2-4aaf-a2a2-2debf3582abe service placement] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:49:58.928678 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d37d5-6891-4101-8990-caf410c573ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e69c0e02ad944f2a85fc5318018e5cb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:49:58.929309 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d37d5-6891-4101-8990-caf410c573ca service swift] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:49:58.929588 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d37d5-6891-4101-8990-caf410c573ca service swift] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:49:58.929856 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d37d5-6891-4101-8990-caf410c573ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:49:58.937746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7d37d5-6891-4101-8990-caf410c573ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:49:58.939278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7d37d5-6891-4101-8990-caf410c573ca service swift] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:49:58.944219 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:49:58.947811 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1392/2792] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 08:49:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 08:50:05.203307 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fe35d-f9c4-4312-9957-e7083a364066 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:50:05.279132 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fe35d-f9c4-4312-9957-e7083a364066 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:50:05.283838 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2eded-2b95-4ee3-aed5-10672cae9523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:50:05.288869 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2eded-2b95-4ee3-aed5-10672cae9523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:05.289322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2eded-2b95-4ee3-aed5-10672cae9523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:05.290129 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2eded-2b95-4ee3-aed5-10672cae9523 admin admin] PATH_INFO: `/v3/users/b603187c9f7346278323ed07a75828ac` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:05.292245 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2eded-2b95-4ee3-aed5-10672cae9523 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b603187c9f7346278323ed07a75828ac)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:50:05.294632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2eded-2b95-4ee3-aed5-10672cae9523 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:50:05.512659 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c7c2eded-2b95-4ee3-aed5-10672cae9523 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b603187c9f7346278323ed07a75828ac'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:05.513482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c7c2eded-2b95-4ee3-aed5-10672cae9523 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b603187c9f7346278323ed07a75828ac'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:05.514314 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c7c2eded-2b95-4ee3-aed5-10672cae9523 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b603187c9f7346278323ed07a75828ac'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:05.543628 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c7c2eded-2b95-4ee3-aed5-10672cae9523 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b603187c9f7346278323ed07a75828ac'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:05.619322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-c7c2eded-2b95-4ee3-aed5-10672cae9523 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b603187c9f7346278323ed07a75828ac'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:05.651454 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1401/2793] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:50:05 2022] DELETE /identity/v3/users/b603187c9f7346278323ed07a75828ac => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:50:05.655293 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d3e72-0cbe-4654-9817-bf421753ecbf None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:05.655293 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d3e72-0cbe-4654-9817-bf421753ecbf None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:05.655293 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d3e72-0cbe-4654-9817-bf421753ecbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:05.716879 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-b49d3e72-0cbe-4654-9817-bf421753ecbf None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:50:05.718961 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b49d3e72-0cbe-4654-9817-bf421753ecbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:50:05.862190 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1393/2794] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:50:05.872522 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2eded-2b95-4ee3-aed5-10672cae9523 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:50:05.874208 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c7c2eded-2b95-4ee3-aed5-10672cae9523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:50:05.998379 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2eded-2b95-4ee3-aed5-10672cae9523 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:50:06.002592 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e35a4-8863-49c6-8572-b90096db08c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:50:06.003321 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e35a4-8863-49c6-8572-b90096db08c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:06.003697 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e35a4-8863-49c6-8572-b90096db08c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:06.004061 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e35a4-8863-49c6-8572-b90096db08c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:06.012071 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:50:06.014759 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000e35a4-8863-49c6-8572-b90096db08c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:50:06.016219 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000e35a4-8863-49c6-8572-b90096db08c8 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:50:06.027236 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1402/2795] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:50:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:50:06.516890 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d3e72-0cbe-4654-9817-bf421753ecbf None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:50:06.518317 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b49d3e72-0cbe-4654-9817-bf421753ecbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:50:06.533503 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d3e72-0cbe-4654-9817-bf421753ecbf None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:50:06.544049 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2969ef8-a517-4928-add6-ef97b7f995d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:50:06.544706 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2969ef8-a517-4928-add6-ef97b7f995d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:06.544976 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2969ef8-a517-4928-add6-ef97b7f995d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:06.545385 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2969ef8-a517-4928-add6-ef97b7f995d5 admin admin] PATH_INFO: `/v3/projects/6a4e7ea0347a480b971c4253f2e4b436` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:06.546635 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2969ef8-a517-4928-add6-ef97b7f995d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a4e7ea0347a480b971c4253f2e4b436)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:50:06.548772 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2969ef8-a517-4928-add6-ef97b7f995d5 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:50:06.649330 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f2969ef8-a517-4928-add6-ef97b7f995d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a4e7ea0347a480b971c4253f2e4b436', 'request_id': 'req-f2969ef8-a517-4928-add6-ef97b7f995d5'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:06.656288 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f2969ef8-a517-4928-add6-ef97b7f995d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a4e7ea0347a480b971c4253f2e4b436', 'request_id': 'req-f2969ef8-a517-4928-add6-ef97b7f995d5'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:06.668019 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f2969ef8-a517-4928-add6-ef97b7f995d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a4e7ea0347a480b971c4253f2e4b436', 'request_id': 'req-f2969ef8-a517-4928-add6-ef97b7f995d5'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:06.750669 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f2969ef8-a517-4928-add6-ef97b7f995d5 admin admin] The token cache is being invalidate because project 6a4e7ea0347a480b971c4253f2e4b436 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:50:06.751295 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f2969ef8-a517-4928-add6-ef97b7f995d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:06.752506 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f2969ef8-a517-4928-add6-ef97b7f995d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:06.755316 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1394/2796] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:50:06 2022] DELETE /identity/v3/projects/6a4e7ea0347a480b971c4253f2e4b436 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:50:06.760731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc75337-4b15-4edc-b260-ae90fb8f66ab None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:06.761369 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc75337-4b15-4edc-b260-ae90fb8f66ab None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:06.761369 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc75337-4b15-4edc-b260-ae90fb8f66ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:06.816305 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-7dc75337-4b15-4edc-b260-ae90fb8f66ab None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:50:06.817553 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7dc75337-4b15-4edc-b260-ae90fb8f66ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:50:06.890869 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1403/2797] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:50:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:50:06.899304 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6709f2-2bc7-4d92-93c0-e911d626bd79 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:06.899518 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6709f2-2bc7-4d92-93c0-e911d626bd79 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:06.899986 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6709f2-2bc7-4d92-93c0-e911d626bd79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:06.920325 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-1c6709f2-2bc7-4d92-93c0-e911d626bd79 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:50:06.921561 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-1c6709f2-2bc7-4d92-93c0-e911d626bd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:50:06.929888 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1395/2798] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:50:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:50:06.934149 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc75337-4b15-4edc-b260-ae90fb8f66ab None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:50:06.944711 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc75337-4b15-4edc-b260-ae90fb8f66ab None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:50:06.949521 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58d161-9c2f-414d-b26a-a49100018e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:50:06.950124 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58d161-9c2f-414d-b26a-a49100018e7f admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:06.950402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58d161-9c2f-414d-b26a-a49100018e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:06.950677 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58d161-9c2f-414d-b26a-a49100018e7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:06.951318 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58d161-9c2f-414d-b26a-a49100018e7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:50:06.951621 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58d161-9c2f-414d-b26a-a49100018e7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:50:06.953240 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58d161-9c2f-414d-b26a-a49100018e7f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:50:06.965651 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1404/2799] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:50:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 08:50:06.970271 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6709f2-2bc7-4d92-93c0-e911d626bd79 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:50:06.980956 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6709f2-2bc7-4d92-93c0-e911d626bd79 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:50:06.986208 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8147bab0-361b-4a0f-99e5-5a2398686326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:50:06.986816 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147bab0-361b-4a0f-99e5-5a2398686326 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:06.987088 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147bab0-361b-4a0f-99e5-5a2398686326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:06.987361 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147bab0-361b-4a0f-99e5-5a2398686326 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:06.987977 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8147bab0-361b-4a0f-99e5-5a2398686326 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:50:06.989896 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8147bab0-361b-4a0f-99e5-5a2398686326 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:50:07.113086 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58d161-9c2f-414d-b26a-a49100018e7f admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:50:07.113991 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1396/2800] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:50:06 2022] POST /identity/v3/projects => generated 373 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:50:07.124125 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58d161-9c2f-414d-b26a-a49100018e7f admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:50:07.193619 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c21ad-4c95-4356-97fb-160ccf23fd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:50:07.194738 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c21ad-4c95-4356-97fb-160ccf23fd71 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:07.195009 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c21ad-4c95-4356-97fb-160ccf23fd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:07.195274 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c21ad-4c95-4356-97fb-160ccf23fd71 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:07.195892 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c21ad-4c95-4356-97fb-160ccf23fd71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:50:07.197823 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c21ad-4c95-4356-97fb-160ccf23fd71 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:50:07.301297 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1405/2801] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:50:07 2022] POST /identity/v3/users => generated 287 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:50:07.302842 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8147bab0-361b-4a0f-99e5-5a2398686326 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:50:07.313341 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8147bab0-361b-4a0f-99e5-5a2398686326 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:50:07.319128 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a6a1b-d8bf-4462-8aba-48529fb00e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:50:07.319733 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a6a1b-d8bf-4462-8aba-48529fb00e31 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:07.320244 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a6a1b-d8bf-4462-8aba-48529fb00e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:07.320560 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a6a1b-d8bf-4462-8aba-48529fb00e31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:07.321303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a6a1b-d8bf-4462-8aba-48529fb00e31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:50:07.322950 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a6a1b-d8bf-4462-8aba-48529fb00e31 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:50:07.341011 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1397/2802] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:50:07.345290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c21ad-4c95-4356-97fb-160ccf23fd71 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:50:07.356490 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c21ad-4c95-4356-97fb-160ccf23fd71 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:50:07.361780 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822b017-8696-4974-bde3-60c8cd876e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:50:07.362400 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822b017-8696-4974-bde3-60c8cd876e9d admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:07.362710 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822b017-8696-4974-bde3-60c8cd876e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:07.362983 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822b017-8696-4974-bde3-60c8cd876e9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:07.363628 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5822b017-8696-4974-bde3-60c8cd876e9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:50:07.365260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5822b017-8696-4974-bde3-60c8cd876e9d admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:50:07.377046 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1406/2803] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:50:07.382329 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a6a1b-d8bf-4462-8aba-48529fb00e31 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:50:07.393554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a6a1b-d8bf-4462-8aba-48529fb00e31 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:50:07.398339 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3bf52e-d7ae-4ab9-a39b-fae4ecd1c619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:50:07.399065 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3bf52e-d7ae-4ab9-a39b-fae4ecd1c619 admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:07.399483 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3bf52e-d7ae-4ab9-a39b-fae4ecd1c619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:07.399814 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3bf52e-d7ae-4ab9-a39b-fae4ecd1c619 admin admin] PATH_INFO: `/v3/projects/1d1e3f37f756438f933dbfea3797e527/users/317ac99b78754c8bab54b882f13661d7/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:07.416186 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3bf52e-d7ae-4ab9-a39b-fae4ecd1c619 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d1e3f37f756438f933dbfea3797e527, user_id=317ac99b78754c8bab54b882f13661d7, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:50:07.418819 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3bf52e-d7ae-4ab9-a39b-fae4ecd1c619 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:50:07.509762 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1398/2804] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:50:07 2022] PUT /identity/v3/projects/1d1e3f37f756438f933dbfea3797e527/users/317ac99b78754c8bab54b882f13661d7/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:50:07.521265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47031e79-a6cd-429b-af91-53b0cf3d5a93 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:07.521551 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47031e79-a6cd-429b-af91-53b0cf3d5a93 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:07.521876 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47031e79-a6cd-429b-af91-53b0cf3d5a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:07.548591 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-47031e79-a6cd-429b-af91-53b0cf3d5a93 None None] MFA Rules not processed for user `317ac99b78754c8bab54b882f13661d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:50:07.549825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-47031e79-a6cd-429b-af91-53b0cf3d5a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:50:07.576111 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:50:07.614169 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1407/2805] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:50:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:50:07.625480 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3bf52e-d7ae-4ab9-a39b-fae4ecd1c619 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:50:07.627215 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-fe3bf52e-d7ae-4ab9-a39b-fae4ecd1c619 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:50:07.645383 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:50:07.706192 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3bf52e-d7ae-4ab9-a39b-fae4ecd1c619 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:50:07.710310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1b07d-a281-4918-9855-4b3636704c3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:50:07.710907 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1b07d-a281-4918-9855-4b3636704c3d service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:07.712520 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1b07d-a281-4918-9855-4b3636704c3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:07.712520 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1b07d-a281-4918-9855-4b3636704c3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:07.719726 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1b07d-a281-4918-9855-4b3636704c3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:50:07.721130 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1b07d-a281-4918-9855-4b3636704c3d service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:50:07.730673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1399/2806] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:50:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:50:16.830217 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47031e79-a6cd-429b-af91-53b0cf3d5a93 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:50:16.831630 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-47031e79-a6cd-429b-af91-53b0cf3d5a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:50:16.895911 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47031e79-a6cd-429b-af91-53b0cf3d5a93 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:50:16.900077 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5d6a9-5176-4f8f-bc27-402813c8cb9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:50:16.900679 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5d6a9-5176-4f8f-bc27-402813c8cb9a service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:16.900952 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5d6a9-5176-4f8f-bc27-402813c8cb9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:16.901271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5d6a9-5176-4f8f-bc27-402813c8cb9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:16.903004 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bed5d6a9-5176-4f8f-bc27-402813c8cb9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:50:16.966412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5d6a9-5176-4f8f-bc27-402813c8cb9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:50:16.967837 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5d6a9-5176-4f8f-bc27-402813c8cb9a service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:50:16.977530 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1408/2807] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:50:16 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 08:50:17.732018 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=29 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:50:19.462105 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1b07d-a281-4918-9855-4b3636704c3d service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:50:19.463855 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-b6f1b07d-a281-4918-9855-4b3636704c3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:50:19.563282 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1b07d-a281-4918-9855-4b3636704c3d service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:50:19.568608 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4afcdb0-662f-4db1-a8aa-ee63b11caa78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:50:19.569485 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afcdb0-662f-4db1-a8aa-ee63b11caa78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:19.569774 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afcdb0-662f-4db1-a8aa-ee63b11caa78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:19.570054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afcdb0-662f-4db1-a8aa-ee63b11caa78 admin admin] PATH_INFO: `/v3/users/fa37eee3ff804505a2fa0891193a2c52` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:19.571174 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4afcdb0-662f-4db1-a8aa-ee63b11caa78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa37eee3ff804505a2fa0891193a2c52)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:50:19.573289 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4afcdb0-662f-4db1-a8aa-ee63b11caa78 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:50:19.698151 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5d6a9-5176-4f8f-bc27-402813c8cb9a service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:50:19.715409 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5d6a9-5176-4f8f-bc27-402813c8cb9a service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:50:19.722355 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c40c12e-0938-4106-8990-d9d21dde9c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:50:19.723008 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40c12e-0938-4106-8990-d9d21dde9c0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:19.723288 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40c12e-0938-4106-8990-d9d21dde9c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:19.723567 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40c12e-0938-4106-8990-d9d21dde9c0b admin admin] PATH_INFO: `/v3/users/317ac99b78754c8bab54b882f13661d7` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:19.725413 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c40c12e-0938-4106-8990-d9d21dde9c0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=317ac99b78754c8bab54b882f13661d7)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:50:19.727342 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c40c12e-0938-4106-8990-d9d21dde9c0b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:50:19.770283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c4afcdb0-662f-4db1-a8aa-ee63b11caa78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa37eee3ff804505a2fa0891193a2c52'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:19.771149 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c4afcdb0-662f-4db1-a8aa-ee63b11caa78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa37eee3ff804505a2fa0891193a2c52'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:19.773337 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c4afcdb0-662f-4db1-a8aa-ee63b11caa78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa37eee3ff804505a2fa0891193a2c52'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:19.781062 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c4afcdb0-662f-4db1-a8aa-ee63b11caa78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa37eee3ff804505a2fa0891193a2c52'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:19.852411 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c4afcdb0-662f-4db1-a8aa-ee63b11caa78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa37eee3ff804505a2fa0891193a2c52'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:19.889589 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1400/2808] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:50:19 2022] DELETE /identity/v3/users/fa37eee3ff804505a2fa0891193a2c52 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:50:19.894369 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8b1cc-1d29-465a-a521-620dcd586971 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:19.894880 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8b1cc-1d29-465a-a521-620dcd586971 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:19.895203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8b1cc-1d29-465a-a521-620dcd586971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:19.923996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-6ac8b1cc-1d29-465a-a521-620dcd586971 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:50:19.924890 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6ac8b1cc-1d29-465a-a521-620dcd586971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:50:19.931155 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2c40c12e-0938-4106-8990-d9d21dde9c0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '317ac99b78754c8bab54b882f13661d7'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:19.931874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2c40c12e-0938-4106-8990-d9d21dde9c0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '317ac99b78754c8bab54b882f13661d7'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:19.945993 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2c40c12e-0938-4106-8990-d9d21dde9c0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '317ac99b78754c8bab54b882f13661d7'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:19.967148 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2c40c12e-0938-4106-8990-d9d21dde9c0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '317ac99b78754c8bab54b882f13661d7'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:19.999050 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1401/2809] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:50:20.000046 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8b1cc-1d29-465a-a521-620dcd586971 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:50:20.001654 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-6ac8b1cc-1d29-465a-a521-620dcd586971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:50:20.027386 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:50:20.057992 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-2c40c12e-0938-4106-8990-d9d21dde9c0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '317ac99b78754c8bab54b882f13661d7'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:20.066112 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:50:20.082703 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8b1cc-1d29-465a-a521-620dcd586971 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:50:20.088857 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f0013-e154-449b-b2d2-0d2b74e180b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:50:20.090671 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f0013-e154-449b-b2d2-0d2b74e180b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:20.091191 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f0013-e154-449b-b2d2-0d2b74e180b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:20.091709 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f0013-e154-449b-b2d2-0d2b74e180b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:20.093212 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1409/2810] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:50:19 2022] DELETE /identity/v3/users/317ac99b78754c8bab54b882f13661d7 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:50:20.099110 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8be37-e35d-4259-a934-6ed0ec2839ae None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:20.099939 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8be37-e35d-4259-a934-6ed0ec2839ae None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:20.100191 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8be37-e35d-4259-a934-6ed0ec2839ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:20.105555 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529f0013-e154-449b-b2d2-0d2b74e180b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:50:20.107418 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529f0013-e154-449b-b2d2-0d2b74e180b1 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:50:20.123699 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1402/2811] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:50:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:50:20.134991 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-f9c8be37-e35d-4259-a934-6ed0ec2839ae None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:50:20.136231 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f9c8be37-e35d-4259-a934-6ed0ec2839ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:50:20.205841 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1410/2812] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:50:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:50:20.221315 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f0013-e154-449b-b2d2-0d2b74e180b1 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:50:20.233837 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f0013-e154-449b-b2d2-0d2b74e180b1 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:50:20.296777 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9b898-34e1-4ad7-90ec-989ff8af711e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:50:20.297451 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9b898-34e1-4ad7-90ec-989ff8af711e service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:20.297730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9b898-34e1-4ad7-90ec-989ff8af711e service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:20.297995 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9b898-34e1-4ad7-90ec-989ff8af711e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:20.314789 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf9b898-34e1-4ad7-90ec-989ff8af711e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:50:20.329489 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf9b898-34e1-4ad7-90ec-989ff8af711e service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:50:20.343332 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1403/2813] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:50:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:50:20.631133 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c8be37-e35d-4259-a934-6ed0ec2839ae None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:50:20.633210 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f9c8be37-e35d-4259-a934-6ed0ec2839ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:50:20.662597 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c8be37-e35d-4259-a934-6ed0ec2839ae None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:50:20.675118 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f234e47d-556d-475b-bd33-d777293aab1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:50:20.677885 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234e47d-556d-475b-bd33-d777293aab1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:20.678179 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234e47d-556d-475b-bd33-d777293aab1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:20.681256 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234e47d-556d-475b-bd33-d777293aab1b admin admin] PATH_INFO: `/v3/projects/b7d1348f5958499b9812242ed59ce2ee` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:20.683733 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f234e47d-556d-475b-bd33-d777293aab1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7d1348f5958499b9812242ed59ce2ee)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:50:20.686367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f234e47d-556d-475b-bd33-d777293aab1b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:50:20.810556 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f234e47d-556d-475b-bd33-d777293aab1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7d1348f5958499b9812242ed59ce2ee', 'request_id': 'req-f234e47d-556d-475b-bd33-d777293aab1b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:20.910103 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f234e47d-556d-475b-bd33-d777293aab1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7d1348f5958499b9812242ed59ce2ee', 'request_id': 'req-f234e47d-556d-475b-bd33-d777293aab1b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:20.918902 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9b898-34e1-4ad7-90ec-989ff8af711e service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:50:20.920432 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-bcf9b898-34e1-4ad7-90ec-989ff8af711e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:50:20.924734 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f234e47d-556d-475b-bd33-d777293aab1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7d1348f5958499b9812242ed59ce2ee', 'request_id': 'req-f234e47d-556d-475b-bd33-d777293aab1b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:20.938419 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f234e47d-556d-475b-bd33-d777293aab1b admin admin] The token cache is being invalidate because project b7d1348f5958499b9812242ed59ce2ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:50:20.942484 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f234e47d-556d-475b-bd33-d777293aab1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:20.942484 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f234e47d-556d-475b-bd33-d777293aab1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:20.942484 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1411/2814] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:50:20 2022] DELETE /identity/v3/projects/b7d1348f5958499b9812242ed59ce2ee => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:50:20.944783 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9b898-34e1-4ad7-90ec-989ff8af711e service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:50:20.947462 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb1ed8-8da9-4a7d-aa9f-5ff49ee27476 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:20.947814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb1ed8-8da9-4a7d-aa9f-5ff49ee27476 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:20.948043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb1ed8-8da9-4a7d-aa9f-5ff49ee27476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:20.970107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-d6cb1ed8-8da9-4a7d-aa9f-5ff49ee27476 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:50:20.971369 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d6cb1ed8-8da9-4a7d-aa9f-5ff49ee27476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:50:21.008969 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3701ec2-b0d0-4a1c-9400-375765401ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:50:21.009763 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3701ec2-b0d0-4a1c-9400-375765401ccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:21.010084 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3701ec2-b0d0-4a1c-9400-375765401ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:21.010335 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3701ec2-b0d0-4a1c-9400-375765401ccf admin admin] PATH_INFO: `/v3/projects/1d1e3f37f756438f933dbfea3797e527` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:21.012029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3701ec2-b0d0-4a1c-9400-375765401ccf admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d1e3f37f756438f933dbfea3797e527)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:50:21.014379 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3701ec2-b0d0-4a1c-9400-375765401ccf admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:50:21.037367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1412/2815] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:50:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:50:21.045167 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3b592-f638-44c1-88b0-3cf84c67d185 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:21.045445 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3b592-f638-44c1-88b0-3cf84c67d185 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:21.045716 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3b592-f638-44c1-88b0-3cf84c67d185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:21.065076 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-83d3b592-f638-44c1-88b0-3cf84c67d185 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:50:21.066290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-83d3b592-f638-44c1-88b0-3cf84c67d185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:50:21.074932 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1413/2816] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 08:50:21.079000 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3b592-f638-44c1-88b0-3cf84c67d185 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:50:21.087852 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3b592-f638-44c1-88b0-3cf84c67d185 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:50:21.092384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a047aa6-eca5-4346-81ac-d954525ec106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:50:21.092960 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a047aa6-eca5-4346-81ac-d954525ec106 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:21.093266 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a047aa6-eca5-4346-81ac-d954525ec106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:21.093535 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a047aa6-eca5-4346-81ac-d954525ec106 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:21.094164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a047aa6-eca5-4346-81ac-d954525ec106 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 08:50:21.094468 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a047aa6-eca5-4346-81ac-d954525ec106 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:50:21.096031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a047aa6-eca5-4346-81ac-d954525ec106 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:50:21.097499 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:50:21.108426 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1414/2817] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 08:50:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 08:50:21.112759 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a047aa6-eca5-4346-81ac-d954525ec106 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:50:21.123774 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a047aa6-eca5-4346-81ac-d954525ec106 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:50:21.125950 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e3701ec2-b0d0-4a1c-9400-375765401ccf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d1e3f37f756438f933dbfea3797e527', 'request_id': 'req-e3701ec2-b0d0-4a1c-9400-375765401ccf'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:21.133559 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e3701ec2-b0d0-4a1c-9400-375765401ccf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d1e3f37f756438f933dbfea3797e527', 'request_id': 'req-e3701ec2-b0d0-4a1c-9400-375765401ccf'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:21.145089 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e3701ec2-b0d0-4a1c-9400-375765401ccf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d1e3f37f756438f933dbfea3797e527', 'request_id': 'req-e3701ec2-b0d0-4a1c-9400-375765401ccf'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:21.146026 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:50:21.193986 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45591794-84c3-4051-b91f-9d68db426b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:50:21.194583 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45591794-84c3-4051-b91f-9d68db426b35 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:21.194861 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45591794-84c3-4051-b91f-9d68db426b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:21.195131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45591794-84c3-4051-b91f-9d68db426b35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:21.195745 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45591794-84c3-4051-b91f-9d68db426b35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:50:21.197661 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45591794-84c3-4051-b91f-9d68db426b35 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:50:21.229861 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e3701ec2-b0d0-4a1c-9400-375765401ccf admin admin] The token cache is being invalidate because project 1d1e3f37f756438f933dbfea3797e527 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:50:21.230301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e3701ec2-b0d0-4a1c-9400-375765401ccf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:21.230924 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-e3701ec2-b0d0-4a1c-9400-375765401ccf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:50:21.232449 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1404/2818] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:50:20 2022] DELETE /identity/v3/projects/1d1e3f37f756438f933dbfea3797e527 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:50:21.296015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1415/2819] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 08:50:21 2022] POST /identity/v3/projects => generated 375 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 08:50:21.300494 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3701ec2-b0d0-4a1c-9400-375765401ccf admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:50:21.302219 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e3701ec2-b0d0-4a1c-9400-375765401ccf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:50:21.316445 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3701ec2-b0d0-4a1c-9400-375765401ccf admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:50:21.377480 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ad540-e819-4f70-bd16-9cc4b0f70afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:50:21.378109 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ad540-e819-4f70-bd16-9cc4b0f70afe admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:21.378395 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ad540-e819-4f70-bd16-9cc4b0f70afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:21.378668 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ad540-e819-4f70-bd16-9cc4b0f70afe admin admin] PATH_INFO: `/v3/users` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:21.379314 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ad540-e819-4f70-bd16-9cc4b0f70afe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:50:21.382251 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ad540-e819-4f70-bd16-9cc4b0f70afe admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:50:21.491884 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1405/2820] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 08:50:21 2022] POST /identity/v3/users => generated 288 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 08:50:21.496220 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45591794-84c3-4051-b91f-9d68db426b35 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:50:21.507332 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45591794-84c3-4051-b91f-9d68db426b35 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:50:21.511873 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af71fdb-ed44-444d-b75d-0bdb376e6e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:50:21.512466 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af71fdb-ed44-444d-b75d-0bdb376e6e5c admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:21.512771 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af71fdb-ed44-444d-b75d-0bdb376e6e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:21.513056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af71fdb-ed44-444d-b75d-0bdb376e6e5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:21.513988 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af71fdb-ed44-444d-b75d-0bdb376e6e5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:50:21.516364 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af71fdb-ed44-444d-b75d-0bdb376e6e5c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:50:21.527108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1416/2821] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 08:50:21.531091 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ad540-e819-4f70-bd16-9cc4b0f70afe admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:50:21.541225 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ad540-e819-4f70-bd16-9cc4b0f70afe admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:50:21.546365 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48163e33-4579-49a0-aa26-b05b80960de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:50:21.549280 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48163e33-4579-49a0-aa26-b05b80960de9 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:21.549280 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48163e33-4579-49a0-aa26-b05b80960de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:21.549280 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48163e33-4579-49a0-aa26-b05b80960de9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:21.549280 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48163e33-4579-49a0-aa26-b05b80960de9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:50:21.549785 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48163e33-4579-49a0-aa26-b05b80960de9 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:50:21.560232 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1406/2822] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 08:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 08:50:21.564388 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af71fdb-ed44-444d-b75d-0bdb376e6e5c admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:50:21.575835 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af71fdb-ed44-444d-b75d-0bdb376e6e5c admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:50:21.580353 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e463e-b20e-48ea-8b81-6b2f8a948fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:50:21.580989 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e463e-b20e-48ea-8b81-6b2f8a948fcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:21.581307 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e463e-b20e-48ea-8b81-6b2f8a948fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:21.581581 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e463e-b20e-48ea-8b81-6b2f8a948fcc admin admin] PATH_INFO: `/v3/projects/2693983a63734f52890ae4bb41c1e915/users/445aaa7a942043bbac9aee5d06b3d092/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:21.596380 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823e463e-b20e-48ea-8b81-6b2f8a948fcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2693983a63734f52890ae4bb41c1e915, user_id=445aaa7a942043bbac9aee5d06b3d092, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:50:21.599059 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823e463e-b20e-48ea-8b81-6b2f8a948fcc admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:50:21.664071 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1417/2823] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 08:50:21 2022] PUT /identity/v3/projects/2693983a63734f52890ae4bb41c1e915/users/445aaa7a942043bbac9aee5d06b3d092/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 08:50:21.675821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da73e3-2516-4cea-9edb-8778ba56fafa None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:21.676044 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da73e3-2516-4cea-9edb-8778ba56fafa None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:21.676318 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da73e3-2516-4cea-9edb-8778ba56fafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:21.705593 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-53da73e3-2516-4cea-9edb-8778ba56fafa None None] MFA Rules not processed for user `445aaa7a942043bbac9aee5d06b3d092`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:50:21.706773 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-53da73e3-2516-4cea-9edb-8778ba56fafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:50:21.771739 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1407/2824] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:50:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:50:21.783006 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e463e-b20e-48ea-8b81-6b2f8a948fcc admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:50:21.784687 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-823e463e-b20e-48ea-8b81-6b2f8a948fcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:50:21.847174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e463e-b20e-48ea-8b81-6b2f8a948fcc admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:50:21.851030 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3019361-b5de-4ec4-9e7e-8bbd8947b861 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:50:21.851629 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3019361-b5de-4ec4-9e7e-8bbd8947b861 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:21.851907 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3019361-b5de-4ec4-9e7e-8bbd8947b861 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:21.852176 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3019361-b5de-4ec4-9e7e-8bbd8947b861 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:21.860821 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3019361-b5de-4ec4-9e7e-8bbd8947b861 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:50:21.862279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3019361-b5de-4ec4-9e7e-8bbd8947b861 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:50:21.872043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1418/2825] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:50:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:50:25.688391 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8c8ba-0bee-49c3-9d48-c92b1b04629a None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:50:25.688864 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8c8ba-0bee-49c3-9d48-c92b1b04629a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:50:25.688970 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8c8ba-0bee-49c3-9d48-c92b1b04629a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:50:25.708069 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-80a8c8ba-0bee-49c3-9d48-c92b1b04629a None None] MFA Rules not processed for user `445aaa7a942043bbac9aee5d06b3d092`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:50:25.709530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-80a8c8ba-0bee-49c3-9d48-c92b1b04629a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:50:25.717324 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1408/2826] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:50:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:50:31.867660 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=77,keystone:INSERT=2 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:50:35.702708 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=71,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:51:14.010930 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3019361-b5de-4ec4-9e7e-8bbd8947b861 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:51:14.013005 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-b3019361-b5de-4ec4-9e7e-8bbd8947b861 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:51:14.070426 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3019361-b5de-4ec4-9e7e-8bbd8947b861 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:51:14.074303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43f247-8c55-48a1-ae1e-65bd26a09d7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:51:14.074896 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43f247-8c55-48a1-ae1e-65bd26a09d7b service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:51:14.075169 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43f247-8c55-48a1-ae1e-65bd26a09d7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:51:14.075438 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43f247-8c55-48a1-ae1e-65bd26a09d7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:51:14.077221 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ce43f247-8c55-48a1-ae1e-65bd26a09d7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:51:14.140983 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce43f247-8c55-48a1-ae1e-65bd26a09d7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:51:14.142437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce43f247-8c55-48a1-ae1e-65bd26a09d7b service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:51:14.152562 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1419/2827] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:51:14 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:51:23.619084 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f8ce5-0473-426e-8bfe-3b738c80929c None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:51:23.619582 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f8ce5-0473-426e-8bfe-3b738c80929c None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:51:23.619709 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f8ce5-0473-426e-8bfe-3b738c80929c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:51:23.638769 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-9a5f8ce5-0473-426e-8bfe-3b738c80929c None None] MFA Rules not processed for user `7f3422c6f9a041ceba851b71bd15d0df`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:51:23.640016 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-9a5f8ce5-0473-426e-8bfe-3b738c80929c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:51:23.695097 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1409/2828] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 08:51:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:51:23.737280 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43f247-8c55-48a1-ae1e-65bd26a09d7b service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:51:23.738654 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ce43f247-8c55-48a1-ae1e-65bd26a09d7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:51:23.806391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43f247-8c55-48a1-ae1e-65bd26a09d7b service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:51:23.811326 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67765a-08ef-4f4d-a217-40921a5b2fdb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e69c0e02ad944f2a85fc5318018e5cb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:51:23.811955 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67765a-08ef-4f4d-a217-40921a5b2fdb service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:51:23.812256 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67765a-08ef-4f4d-a217-40921a5b2fdb service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:51:23.812527 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67765a-08ef-4f4d-a217-40921a5b2fdb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:51:23.820263 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd67765a-08ef-4f4d-a217-40921a5b2fdb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:51:23.821920 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd67765a-08ef-4f4d-a217-40921a5b2fdb service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:51:23.829723 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1420/2829] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 08:51:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 08:51:24.581999 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce57fd-e6e5-4f02-b95d-8326c607f600 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:51:24.582288 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce57fd-e6e5-4f02-b95d-8326c607f600 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:51:24.582562 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce57fd-e6e5-4f02-b95d-8326c607f600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:51:24.602288 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-89ce57fd-e6e5-4f02-b95d-8326c607f600 None None] MFA Rules not processed for user `7f3422c6f9a041ceba851b71bd15d0df`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:51:24.604128 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-89ce57fd-e6e5-4f02-b95d-8326c607f600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:51:24.611923 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1410/2830] 10.222.0.33 () {58 vars in 967 bytes} [Mon Aug 8 08:51:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:51:24.624031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67765a-08ef-4f4d-a217-40921a5b2fdb service swift] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:51:24.635180 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67765a-08ef-4f4d-a217-40921a5b2fdb service swift] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:51:24.640302 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a26d9-e2ae-4172-9056-baa07490cb74 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e69c0e02ad944f2a85fc5318018e5cb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:51:24.640896 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a26d9-e2ae-4172-9056-baa07490cb74 service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:51:24.641228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a26d9-e2ae-4172-9056-baa07490cb74 service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:51:24.641498 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a26d9-e2ae-4172-9056-baa07490cb74 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:51:24.650119 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a26d9-e2ae-4172-9056-baa07490cb74 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:51:24.651786 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a26d9-e2ae-4172-9056-baa07490cb74 service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:51:24.659585 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1421/2831] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 08:51:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 08:51:25.208942 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0416e-9512-4ad5-a36a-48b01befb23a None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:51:25.210264 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0416e-9512-4ad5-a36a-48b01befb23a None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:51:25.210264 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0416e-9512-4ad5-a36a-48b01befb23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:51:25.229080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-25a0416e-9512-4ad5-a36a-48b01befb23a None None] MFA Rules not processed for user `7f3422c6f9a041ceba851b71bd15d0df`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:51:25.230545 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-25a0416e-9512-4ad5-a36a-48b01befb23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:51:25.237471 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1411/2832] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 08:51:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:51:25.249402 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a26d9-e2ae-4172-9056-baa07490cb74 service swift] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:51:25.260138 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a26d9-e2ae-4172-9056-baa07490cb74 service swift] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:51:25.265274 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d33260-8a34-4947-857c-4941f866f573 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e69c0e02ad944f2a85fc5318018e5cb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:51:25.265876 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d33260-8a34-4947-857c-4941f866f573 service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:51:25.266154 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d33260-8a34-4947-857c-4941f866f573 service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:51:25.266421 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d33260-8a34-4947-857c-4941f866f573 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:51:25.275032 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d33260-8a34-4947-857c-4941f866f573 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:51:25.276625 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d33260-8a34-4947-857c-4941f866f573 service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:51:25.284381 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1422/2833] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 08:51:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 08:51:25.878397 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0346175d-3c09-4a4c-8b30-7e23a06ed005 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:51:25.878680 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0346175d-3c09-4a4c-8b30-7e23a06ed005 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:51:25.879011 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0346175d-3c09-4a4c-8b30-7e23a06ed005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:51:25.898409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-0346175d-3c09-4a4c-8b30-7e23a06ed005 None None] MFA Rules not processed for user `7f3422c6f9a041ceba851b71bd15d0df`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:51:25.900403 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0346175d-3c09-4a4c-8b30-7e23a06ed005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:51:25.907637 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1412/2834] 10.222.0.33 () {58 vars in 967 bytes} [Mon Aug 8 08:51:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:51:25.919671 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d33260-8a34-4947-857c-4941f866f573 service swift] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:51:25.930019 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d33260-8a34-4947-857c-4941f866f573 service swift] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:51:25.935227 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b431c-0a94-431e-8737-0e790a849427 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e69c0e02ad944f2a85fc5318018e5cb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:51:25.935859 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b431c-0a94-431e-8737-0e790a849427 service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:51:25.936137 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b431c-0a94-431e-8737-0e790a849427 service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:51:25.936404 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b431c-0a94-431e-8737-0e790a849427 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:51:25.944281 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b431c-0a94-431e-8737-0e790a849427 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:51:25.945884 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b431c-0a94-431e-8737-0e790a849427 service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:51:25.950804 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=100 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:51:25.954353 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1423/2835] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 08:51:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 08:51:30.448821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0346175d-3c09-4a4c-8b30-7e23a06ed005 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:51:30.450269 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0346175d-3c09-4a4c-8b30-7e23a06ed005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:51:30.462784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0346175d-3c09-4a4c-8b30-7e23a06ed005 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:51:30.466375 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c340e-a266-4550-8ae6-b40953c5dba7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:51:30.467059 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c340e-a266-4550-8ae6-b40953c5dba7 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:51:30.467391 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c340e-a266-4550-8ae6-b40953c5dba7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:51:30.467676 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c340e-a266-4550-8ae6-b40953c5dba7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:51:30.469414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d93c340e-a266-4550-8ae6-b40953c5dba7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:51:30.551112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93c340e-a266-4550-8ae6-b40953c5dba7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:51:30.552535 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93c340e-a266-4550-8ae6-b40953c5dba7 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:51:30.573454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1413/2836] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:51:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:51:31.761653 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b431c-0a94-431e-8737-0e790a849427 service swift] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:51:31.770862 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b431c-0a94-431e-8737-0e790a849427 service swift] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:51:31.774425 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2e8c7-748a-46ba-b2f8-90ce7443afc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:51:31.775009 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2e8c7-748a-46ba-b2f8-90ce7443afc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:51:31.775283 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2e8c7-748a-46ba-b2f8-90ce7443afc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:51:31.775550 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2e8c7-748a-46ba-b2f8-90ce7443afc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:51:31.777265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f2f2e8c7-748a-46ba-b2f8-90ce7443afc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:51:31.790636 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f2e8c7-748a-46ba-b2f8-90ce7443afc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:51:31.792043 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f2e8c7-748a-46ba-b2f8-90ce7443afc6 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:51:31.802730 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1424/2837] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:51:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:51:40.570278 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=77 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:51:41.797979 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=6 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:52:23.294759 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c340e-a266-4550-8ae6-b40953c5dba7 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:52:23.296196 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d93c340e-a266-4550-8ae6-b40953c5dba7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:52:23.311707 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c340e-a266-4550-8ae6-b40953c5dba7 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:52:23.318054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58abe54-075a-461a-9740-5bd8865caa4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:52:23.319998 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58abe54-075a-461a-9740-5bd8865caa4e service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:52:23.321107 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58abe54-075a-461a-9740-5bd8865caa4e service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:52:23.332773 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58abe54-075a-461a-9740-5bd8865caa4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:52:23.339547 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d58abe54-075a-461a-9740-5bd8865caa4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:52:23.383787 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58abe54-075a-461a-9740-5bd8865caa4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:52:23.391157 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58abe54-075a-461a-9740-5bd8865caa4e service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:52:23.418192 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1414/2838] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:52:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:52:23.507617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2e8c7-748a-46ba-b2f8-90ce7443afc6 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:52:23.516768 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2e8c7-748a-46ba-b2f8-90ce7443afc6 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:52:23.520677 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34fe940-68f0-43f0-b513-4d85af281b65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:52:23.522603 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34fe940-68f0-43f0-b513-4d85af281b65 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:52:23.522948 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34fe940-68f0-43f0-b513-4d85af281b65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:52:23.523285 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34fe940-68f0-43f0-b513-4d85af281b65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:52:23.525496 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c34fe940-68f0-43f0-b513-4d85af281b65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:52:23.535779 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34fe940-68f0-43f0-b513-4d85af281b65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:52:23.541907 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34fe940-68f0-43f0-b513-4d85af281b65 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:52:23.546274 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1425/2839] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:52:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:52:33.406434 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=6 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:52:33.542393 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=6 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:54:25.969440 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58abe54-075a-461a-9740-5bd8865caa4e service nova] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:54:25.970926 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d58abe54-075a-461a-9740-5bd8865caa4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:54:25.983609 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58abe54-075a-461a-9740-5bd8865caa4e service nova] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:54:25.987035 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4608a31-d368-442c-854e-9021c5754935 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:54:25.987632 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4608a31-d368-442c-854e-9021c5754935 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:54:25.987916 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4608a31-d368-442c-854e-9021c5754935 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:54:25.988186 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4608a31-d368-442c-854e-9021c5754935 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:54:25.989932 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d4608a31-d368-442c-854e-9021c5754935 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:54:26.049699 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4608a31-d368-442c-854e-9021c5754935 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:54:26.051133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4608a31-d368-442c-854e-9021c5754935 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:54:26.059911 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1415/2840] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:54:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:54:31.461726 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34fe940-68f0-43f0-b513-4d85af281b65 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:54:31.474903 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34fe940-68f0-43f0-b513-4d85af281b65 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:54:31.480564 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6db866-3740-4a8b-a9a3-882df936af41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:54:31.481173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6db866-3740-4a8b-a9a3-882df936af41 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:54:31.481466 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6db866-3740-4a8b-a9a3-882df936af41 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:54:31.481736 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6db866-3740-4a8b-a9a3-882df936af41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:54:31.483686 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ae6db866-3740-4a8b-a9a3-882df936af41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:54:31.561365 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6db866-3740-4a8b-a9a3-882df936af41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:54:31.562869 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6db866-3740-4a8b-a9a3-882df936af41 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:54:31.581592 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1426/2841] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:54:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:54:36.055889 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=28 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:54:36.854511 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4608a31-d368-442c-854e-9021c5754935 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:54:36.855854 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d4608a31-d368-442c-854e-9021c5754935 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:54:36.872117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4608a31-d368-442c-854e-9021c5754935 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:54:36.877712 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312626f9-d489-4395-ba6c-7e1f56f4ec39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:54:36.878312 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312626f9-d489-4395-ba6c-7e1f56f4ec39 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:54:36.878593 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312626f9-d489-4395-ba6c-7e1f56f4ec39 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:54:36.878891 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312626f9-d489-4395-ba6c-7e1f56f4ec39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:54:36.880600 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-312626f9-d489-4395-ba6c-7e1f56f4ec39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:54:36.936944 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312626f9-d489-4395-ba6c-7e1f56f4ec39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:54:36.938552 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312626f9-d489-4395-ba6c-7e1f56f4ec39 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:54:36.947394 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1416/2842] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:54:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:54:41.578744 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=41 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:54:46.943311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=28 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:55:48.250731 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6db866-3740-4a8b-a9a3-882df936af41 service nova] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:55:48.252104 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ae6db866-3740-4a8b-a9a3-882df936af41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:55:48.320875 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6db866-3740-4a8b-a9a3-882df936af41 service nova] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:55:48.329052 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08942681-f27a-4504-91e8-85661d4ba403 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c23a782b9b4144bace1248c3b2e754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:55:48.329679 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08942681-f27a-4504-91e8-85661d4ba403 service placement] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:55:48.329955 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08942681-f27a-4504-91e8-85661d4ba403 service placement] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:55:48.330224 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08942681-f27a-4504-91e8-85661d4ba403 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:55:48.331928 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-08942681-f27a-4504-91e8-85661d4ba403 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:55:48.343978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08942681-f27a-4504-91e8-85661d4ba403 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:55:48.345607 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08942681-f27a-4504-91e8-85661d4ba403 service placement] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:55:48.355393 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1427/2843] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:55:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:55:58.351718 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=32 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:56:02.555163 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312626f9-d489-4395-ba6c-7e1f56f4ec39 service nova] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:56:02.564488 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312626f9-d489-4395-ba6c-7e1f56f4ec39 service nova] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:56:02.568033 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5735dba3-69d8-4bff-94f1-05d5f6df5fde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:56:02.568621 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735dba3-69d8-4bff-94f1-05d5f6df5fde service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:56:02.568898 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735dba3-69d8-4bff-94f1-05d5f6df5fde service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:56:02.569203 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735dba3-69d8-4bff-94f1-05d5f6df5fde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:56:02.577258 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5735dba3-69d8-4bff-94f1-05d5f6df5fde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:56:02.578666 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5735dba3-69d8-4bff-94f1-05d5f6df5fde service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:56:02.587410 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1417/2844] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:56:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:56:12.583380 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=6 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:56:30.835557 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08942681-f27a-4504-91e8-85661d4ba403 service placement] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:56:30.844795 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08942681-f27a-4504-91e8-85661d4ba403 service placement] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:56:30.848321 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae1385-4291-4a21-9ad3-6c97cc1523dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:56:30.848903 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae1385-4291-4a21-9ad3-6c97cc1523dd service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:56:30.849206 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae1385-4291-4a21-9ad3-6c97cc1523dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:56:30.849475 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae1385-4291-4a21-9ad3-6c97cc1523dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:56:30.860174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fae1385-4291-4a21-9ad3-6c97cc1523dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:56:30.861598 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fae1385-4291-4a21-9ad3-6c97cc1523dd service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:56:30.873023 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1428/2845] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:56:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:56:40.868239 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=6 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:57:53.895283 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5735dba3-69d8-4bff-94f1-05d5f6df5fde service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:57:53.904746 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5735dba3-69d8-4bff-94f1-05d5f6df5fde service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:57:53.909397 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eba5c3-2dd1-4098-bf85-abaf089d205c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:57:53.910028 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eba5c3-2dd1-4098-bf85-abaf089d205c service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:57:53.910307 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eba5c3-2dd1-4098-bf85-abaf089d205c service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:57:53.910577 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eba5c3-2dd1-4098-bf85-abaf089d205c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:57:53.918284 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eba5c3-2dd1-4098-bf85-abaf089d205c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:57:53.919689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eba5c3-2dd1-4098-bf85-abaf089d205c service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:57:53.928167 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1418/2846] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:57:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:58:02.735916 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae1385-4291-4a21-9ad3-6c97cc1523dd service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:58:02.746140 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae1385-4291-4a21-9ad3-6c97cc1523dd service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:58:02.749771 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c836aaa-c10d-4138-9fe1-27eef387707f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:58:02.750360 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c836aaa-c10d-4138-9fe1-27eef387707f service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:58:02.750636 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c836aaa-c10d-4138-9fe1-27eef387707f service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:58:02.750905 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c836aaa-c10d-4138-9fe1-27eef387707f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:58:02.761406 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c836aaa-c10d-4138-9fe1-27eef387707f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:58:02.762854 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c836aaa-c10d-4138-9fe1-27eef387707f service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:58:02.774924 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1429/2847] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:58:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:58:02.952789 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eba5c3-2dd1-4098-bf85-abaf089d205c service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:58:03.018398 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eba5c3-2dd1-4098-bf85-abaf089d205c service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:58:03.022781 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c3ed7-d067-4df4-a9ab-949fef4f3f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:58:03.023572 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c3ed7-d067-4df4-a9ab-949fef4f3f2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:58:03.024309 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c3ed7-d067-4df4-a9ab-949fef4f3f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:58:03.024619 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c3ed7-d067-4df4-a9ab-949fef4f3f2b admin admin] PATH_INFO: `/v3/users/445aaa7a942043bbac9aee5d06b3d092` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:58:03.025716 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c3ed7-d067-4df4-a9ab-949fef4f3f2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=445aaa7a942043bbac9aee5d06b3d092)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:58:03.027616 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c3ed7-d067-4df4-a9ab-949fef4f3f2b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:58:03.217234 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-477c3ed7-d067-4df4-a9ab-949fef4f3f2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '445aaa7a942043bbac9aee5d06b3d092'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:58:03.217898 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-477c3ed7-d067-4df4-a9ab-949fef4f3f2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '445aaa7a942043bbac9aee5d06b3d092'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:58:03.218502 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-477c3ed7-d067-4df4-a9ab-949fef4f3f2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '445aaa7a942043bbac9aee5d06b3d092'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:58:03.225423 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-477c3ed7-d067-4df4-a9ab-949fef4f3f2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '445aaa7a942043bbac9aee5d06b3d092'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:58:03.306569 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-477c3ed7-d067-4df4-a9ab-949fef4f3f2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '445aaa7a942043bbac9aee5d06b3d092'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:58:03.324654 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1419/2848] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 08:58:02 2022] DELETE /identity/v3/users/445aaa7a942043bbac9aee5d06b3d092 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:58:03.329052 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f54de-4e42-4277-be31-ffb1b1486897 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:58:03.329371 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f54de-4e42-4277-be31-ffb1b1486897 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:58:03.329648 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f54de-4e42-4277-be31-ffb1b1486897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:58:03.348305 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-301f54de-4e42-4277-be31-ffb1b1486897 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 08:58:03.349521 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-301f54de-4e42-4277-be31-ffb1b1486897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:58:03.408098 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1430/2849] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 08:58:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 08:58:03.418202 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c3ed7-d067-4df4-a9ab-949fef4f3f2b admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:58:03.420681 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-477c3ed7-d067-4df4-a9ab-949fef4f3f2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:58:03.476497 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c3ed7-d067-4df4-a9ab-949fef4f3f2b admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:58:03.480141 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0720b7-3ec6-4e05-8a38-61ca4e2683db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:58:03.480733 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0720b7-3ec6-4e05-8a38-61ca4e2683db service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:58:03.481011 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0720b7-3ec6-4e05-8a38-61ca4e2683db service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:58:03.481314 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0720b7-3ec6-4e05-8a38-61ca4e2683db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:58:03.490666 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0720b7-3ec6-4e05-8a38-61ca4e2683db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:58:03.494897 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0720b7-3ec6-4e05-8a38-61ca4e2683db service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:58:03.502383 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1420/2850] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 08:58:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 08:58:03.791027 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f54de-4e42-4277-be31-ffb1b1486897 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 08:58:03.792448 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-301f54de-4e42-4277-be31-ffb1b1486897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 08:58:03.804311 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f54de-4e42-4277-be31-ffb1b1486897 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 08:58:03.809985 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4ee11-1cf1-42ae-adb2-7b52901ffd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 08:58:03.810919 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4ee11-1cf1-42ae-adb2-7b52901ffd0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 08:58:03.811381 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4ee11-1cf1-42ae-adb2-7b52901ffd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 08:58:03.811568 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4ee11-1cf1-42ae-adb2-7b52901ffd0b admin admin] PATH_INFO: `/v3/projects/2693983a63734f52890ae4bb41c1e915` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 08:58:03.812708 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4ee11-1cf1-42ae-adb2-7b52901ffd0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2693983a63734f52890ae4bb41c1e915)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 08:58:03.815158 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4ee11-1cf1-42ae-adb2-7b52901ffd0b admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 08:58:03.921502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7fe4ee11-1cf1-42ae-adb2-7b52901ffd0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2693983a63734f52890ae4bb41c1e915', 'request_id': 'req-7fe4ee11-1cf1-42ae-adb2-7b52901ffd0b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:58:04.012271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7fe4ee11-1cf1-42ae-adb2-7b52901ffd0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2693983a63734f52890ae4bb41c1e915', 'request_id': 'req-7fe4ee11-1cf1-42ae-adb2-7b52901ffd0b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:58:04.019228 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7fe4ee11-1cf1-42ae-adb2-7b52901ffd0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2693983a63734f52890ae4bb41c1e915', 'request_id': 'req-7fe4ee11-1cf1-42ae-adb2-7b52901ffd0b'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:58:04.029877 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7fe4ee11-1cf1-42ae-adb2-7b52901ffd0b admin admin] The token cache is being invalidate because project 2693983a63734f52890ae4bb41c1e915 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 08:58:04.030159 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7fe4ee11-1cf1-42ae-adb2-7b52901ffd0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:58:04.030778 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-7fe4ee11-1cf1-42ae-adb2-7b52901ffd0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 08:58:04.032406 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1431/2851] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 08:58:03 2022] DELETE /identity/v3/projects/2693983a63734f52890ae4bb41c1e915 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 08:58:13.498078 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 08:58:14.026847 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:00:02.970797 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0720b7-3ec6-4e05-8a38-61ca4e2683db service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:00:02.972246 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-4a0720b7-3ec6-4e05-8a38-61ca4e2683db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:00:03.057697 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0720b7-3ec6-4e05-8a38-61ca4e2683db service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:00:03.104839 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff1a73c-70fa-450b-a317-1f848cc47076 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:00:03.105473 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1a73c-70fa-450b-a317-1f848cc47076 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:00:03.105754 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1a73c-70fa-450b-a317-1f848cc47076 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:00:03.106025 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1a73c-70fa-450b-a317-1f848cc47076 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:00:03.107740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8ff1a73c-70fa-450b-a317-1f848cc47076 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:00:03.178335 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff1a73c-70fa-450b-a317-1f848cc47076 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:00:03.179788 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff1a73c-70fa-450b-a317-1f848cc47076 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:00:03.189393 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1421/2852] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:00:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 09:00:03.237617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4ee11-1cf1-42ae-adb2-7b52901ffd0b admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:00:03.238998 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7fe4ee11-1cf1-42ae-adb2-7b52901ffd0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:00:03.255111 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4ee11-1cf1-42ae-adb2-7b52901ffd0b admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:00:03.258854 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df18241-9132-4214-88fe-44f7d8c35866 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:00:03.259457 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df18241-9132-4214-88fe-44f7d8c35866 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:00:03.259734 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df18241-9132-4214-88fe-44f7d8c35866 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:00:03.260008 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df18241-9132-4214-88fe-44f7d8c35866 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:00:03.261843 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5df18241-9132-4214-88fe-44f7d8c35866 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:00:03.340772 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df18241-9132-4214-88fe-44f7d8c35866 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:00:03.342224 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df18241-9132-4214-88fe-44f7d8c35866 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:00:03.360404 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1432/2853] 10.222.0.33 () {60 vars in 1396 bytes} [Mon Aug 8 09:00:03 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:00:03.413740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff1a73c-70fa-450b-a317-1f848cc47076 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:00:03.424170 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff1a73c-70fa-450b-a317-1f848cc47076 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:00:03.427754 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e698f5b-31e1-4c79-aca2-6e117ad14d71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:00:03.428348 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e698f5b-31e1-4c79-aca2-6e117ad14d71 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:00:03.428620 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e698f5b-31e1-4c79-aca2-6e117ad14d71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:00:03.428885 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e698f5b-31e1-4c79-aca2-6e117ad14d71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:00:03.430789 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-3e698f5b-31e1-4c79-aca2-6e117ad14d71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:00:03.457917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:00:03.508459 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e698f5b-31e1-4c79-aca2-6e117ad14d71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:00:03.510232 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e698f5b-31e1-4c79-aca2-6e117ad14d71 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:00:03.528200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1422/2854] 10.222.0.33 () {60 vars in 1396 bytes} [Mon Aug 8 09:00:03 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:00:13.357921 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=43 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:00:13.525740 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=28 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:00:52.266456 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df18241-9132-4214-88fe-44f7d8c35866 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:00:52.267826 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5df18241-9132-4214-88fe-44f7d8c35866 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:00:52.363685 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df18241-9132-4214-88fe-44f7d8c35866 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:00:52.369442 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f7311-186e-486d-9ed2-77a812983905 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c23a782b9b4144bace1248c3b2e754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:00:52.370068 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f7311-186e-486d-9ed2-77a812983905 service placement] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:00:52.370343 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f7311-186e-486d-9ed2-77a812983905 service placement] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:00:52.370617 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f7311-186e-486d-9ed2-77a812983905 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:00:52.372307 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f94f7311-186e-486d-9ed2-77a812983905 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:00:52.383671 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f7311-186e-486d-9ed2-77a812983905 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:00:52.385335 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f7311-186e-486d-9ed2-77a812983905 service placement] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:00:52.395015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1433/2855] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:00:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:01:02.390839 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=44 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:02:03.642416 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e698f5b-31e1-4c79-aca2-6e117ad14d71 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:02:03.660979 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e698f5b-31e1-4c79-aca2-6e117ad14d71 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:02:03.666043 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022087f8-343d-45dd-b290-894eb7e3188a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:02:03.666803 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022087f8-343d-45dd-b290-894eb7e3188a service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:02:03.667060 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022087f8-343d-45dd-b290-894eb7e3188a service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:02:03.672079 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022087f8-343d-45dd-b290-894eb7e3188a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:02:03.672079 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-022087f8-343d-45dd-b290-894eb7e3188a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:02:03.758646 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022087f8-343d-45dd-b290-894eb7e3188a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:02:03.760716 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022087f8-343d-45dd-b290-894eb7e3188a service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:02:03.781613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1423/2856] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:02:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:02:13.778917 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=50 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:02:56.911603 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f7311-186e-486d-9ed2-77a812983905 service placement] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:02:56.913119 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f94f7311-186e-486d-9ed2-77a812983905 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:02:56.989618 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f7311-186e-486d-9ed2-77a812983905 service placement] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:02:56.993319 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24074d2f-a63d-4c38-9ee7-e48e70749bef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:02:56.994474 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24074d2f-a63d-4c38-9ee7-e48e70749bef service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:02:56.994779 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24074d2f-a63d-4c38-9ee7-e48e70749bef service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:02:56.995303 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24074d2f-a63d-4c38-9ee7-e48e70749bef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:02:56.997038 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-24074d2f-a63d-4c38-9ee7-e48e70749bef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:02:57.006703 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24074d2f-a63d-4c38-9ee7-e48e70749bef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:02:57.008131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24074d2f-a63d-4c38-9ee7-e48e70749bef service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:02:57.016814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1434/2857] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:02:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:03:07.013259 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=35 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:05:03.310533 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022087f8-343d-45dd-b290-894eb7e3188a service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:05:03.320270 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022087f8-343d-45dd-b290-894eb7e3188a service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:05:03.323854 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd49b8-8b43-4c9f-8bf7-801f10915135 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:05:03.324450 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd49b8-8b43-4c9f-8bf7-801f10915135 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:05:03.324730 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd49b8-8b43-4c9f-8bf7-801f10915135 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:05:03.325005 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd49b8-8b43-4c9f-8bf7-801f10915135 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:05:03.333344 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd49b8-8b43-4c9f-8bf7-801f10915135 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:05:03.334759 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd49b8-8b43-4c9f-8bf7-801f10915135 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:05:03.343847 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1424/2858] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:05:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:05:03.393100 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24074d2f-a63d-4c38-9ee7-e48e70749bef service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:05:03.402350 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24074d2f-a63d-4c38-9ee7-e48e70749bef service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:05:03.406690 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34228b-2e38-4ece-b7bd-115e3b342118 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:05:03.407326 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34228b-2e38-4ece-b7bd-115e3b342118 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:05:03.407606 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34228b-2e38-4ece-b7bd-115e3b342118 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:05:03.407876 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34228b-2e38-4ece-b7bd-115e3b342118 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:05:03.416830 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f34228b-2e38-4ece-b7bd-115e3b342118 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:05:03.418315 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f34228b-2e38-4ece-b7bd-115e3b342118 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:05:03.428146 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1435/2859] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:05:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:05:03.716223 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd49b8-8b43-4c9f-8bf7-801f10915135 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:05:03.725664 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd49b8-8b43-4c9f-8bf7-801f10915135 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:05:03.729378 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef8ac8-ad8c-42af-acb7-af2d4386278d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:05:03.729968 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef8ac8-ad8c-42af-acb7-af2d4386278d service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:05:03.730248 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef8ac8-ad8c-42af-acb7-af2d4386278d service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:05:03.730547 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef8ac8-ad8c-42af-acb7-af2d4386278d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:05:03.740073 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef8ac8-ad8c-42af-acb7-af2d4386278d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:05:03.741526 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef8ac8-ad8c-42af-acb7-af2d4386278d service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:05:03.769208 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1425/2860] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:05:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:05:13.423915 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=6 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:05:13.766690 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=19 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:05:57.222268 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34228b-2e38-4ece-b7bd-115e3b342118 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:05:57.233870 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34228b-2e38-4ece-b7bd-115e3b342118 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:05:57.239925 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb26f1a-ac0b-4253-8e35-c71858ebd7ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c23a782b9b4144bace1248c3b2e754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:05:57.240541 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb26f1a-ac0b-4253-8e35-c71858ebd7ad service placement] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:05:57.240825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb26f1a-ac0b-4253-8e35-c71858ebd7ad service placement] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:05:57.241104 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb26f1a-ac0b-4253-8e35-c71858ebd7ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:05:57.250212 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb26f1a-ac0b-4253-8e35-c71858ebd7ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:05:57.252036 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb26f1a-ac0b-4253-8e35-c71858ebd7ad service placement] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:05:57.261850 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1436/2861] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:05:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:06:07.257587 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=6 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:08:03.903022 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef8ac8-ad8c-42af-acb7-af2d4386278d service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:08:03.913242 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef8ac8-ad8c-42af-acb7-af2d4386278d service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:08:03.917200 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b237cd7-4f12-402d-a458-76a651a8de45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:08:03.917804 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b237cd7-4f12-402d-a458-76a651a8de45 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:08:03.918084 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b237cd7-4f12-402d-a458-76a651a8de45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:08:03.918386 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b237cd7-4f12-402d-a458-76a651a8de45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:08:03.925691 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b237cd7-4f12-402d-a458-76a651a8de45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:08:03.927051 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b237cd7-4f12-402d-a458-76a651a8de45 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:08:03.935290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1426/2862] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:08:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:08:04.672083 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb26f1a-ac0b-4253-8e35-c71858ebd7ad service placement] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:08:04.683841 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb26f1a-ac0b-4253-8e35-c71858ebd7ad service placement] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:08:04.694302 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf69318-0795-4914-9466-6f5492357158 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:08:04.695192 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf69318-0795-4914-9466-6f5492357158 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:08:04.695543 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf69318-0795-4914-9466-6f5492357158 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:08:04.695888 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf69318-0795-4914-9466-6f5492357158 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:08:04.696025 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b237cd7-4f12-402d-a458-76a651a8de45 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:08:04.705029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b237cd7-4f12-402d-a458-76a651a8de45 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:08:04.707457 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf69318-0795-4914-9466-6f5492357158 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:08:04.708986 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8de5db-8939-4322-9e06-7b2a21c9438e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:08:04.709559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf69318-0795-4914-9466-6f5492357158 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:08:04.709962 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8de5db-8939-4322-9e06-7b2a21c9438e service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:08:04.709962 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8de5db-8939-4322-9e06-7b2a21c9438e service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:08:04.710197 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8de5db-8939-4322-9e06-7b2a21c9438e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:08:04.711966 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-bc8de5db-8939-4322-9e06-7b2a21c9438e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:08:04.722427 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1437/2863] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:08:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:08:04.727133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8de5db-8939-4322-9e06-7b2a21c9438e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:08:04.728604 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8de5db-8939-4322-9e06-7b2a21c9438e service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:08:04.740483 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1427/2864] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:08:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 09:08:05.857371 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf69318-0795-4914-9466-6f5492357158 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:08:05.859965 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-faf69318-0795-4914-9466-6f5492357158 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:08:05.875320 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf69318-0795-4914-9466-6f5492357158 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:08:05.881266 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f3e60-4a74-4c97-a039-526d0573c11a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:08:05.881998 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f3e60-4a74-4c97-a039-526d0573c11a service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:08:05.882279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f3e60-4a74-4c97-a039-526d0573c11a service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:08:05.882558 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f3e60-4a74-4c97-a039-526d0573c11a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:08:05.884267 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-788f3e60-4a74-4c97-a039-526d0573c11a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:08:05.896367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f3e60-4a74-4c97-a039-526d0573c11a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:08:05.898242 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f3e60-4a74-4c97-a039-526d0573c11a service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:08:05.909327 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1438/2865] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:08:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:08:14.735311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=12 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:08:15.904444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=12 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:10:03.480135 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8de5db-8939-4322-9e06-7b2a21c9438e service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:10:03.481568 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-bc8de5db-8939-4322-9e06-7b2a21c9438e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:10:03.537305 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8de5db-8939-4322-9e06-7b2a21c9438e service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:10:03.543467 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d0f21-c2f9-4cda-85b4-aae6762ee71f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:10:03.544092 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d0f21-c2f9-4cda-85b4-aae6762ee71f service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:10:03.544381 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d0f21-c2f9-4cda-85b4-aae6762ee71f service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:10:03.544652 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d0f21-c2f9-4cda-85b4-aae6762ee71f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:10:03.546449 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-df1d0f21-c2f9-4cda-85b4-aae6762ee71f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:10:03.624094 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d0f21-c2f9-4cda-85b4-aae6762ee71f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:10:03.625684 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d0f21-c2f9-4cda-85b4-aae6762ee71f service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:10:03.643850 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1428/2866] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:10:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:10:04.210442 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f3e60-4a74-4c97-a039-526d0573c11a service nova] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:10:04.244654 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f3e60-4a74-4c97-a039-526d0573c11a service nova] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:10:04.250155 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bce6f4-2895-4214-96f4-1122fd6875c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:10:04.250748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bce6f4-2895-4214-96f4-1122fd6875c7 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:10:04.251027 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bce6f4-2895-4214-96f4-1122fd6875c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:10:04.251297 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bce6f4-2895-4214-96f4-1122fd6875c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:10:04.253082 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-37bce6f4-2895-4214-96f4-1122fd6875c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:10:04.331327 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bce6f4-2895-4214-96f4-1122fd6875c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:10:04.332900 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bce6f4-2895-4214-96f4-1122fd6875c7 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:10:04.351085 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1439/2867] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:10:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:10:06.098445 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d0f21-c2f9-4cda-85b4-aae6762ee71f service nova] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:10:06.098445 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-df1d0f21-c2f9-4cda-85b4-aae6762ee71f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:10:06.163726 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d0f21-c2f9-4cda-85b4-aae6762ee71f service nova] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:10:06.167298 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99190f-9998-4836-bc42-d2a6852ac516 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:10:06.167891 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99190f-9998-4836-bc42-d2a6852ac516 service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:10:06.168166 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99190f-9998-4836-bc42-d2a6852ac516 service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:10:06.168431 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99190f-9998-4836-bc42-d2a6852ac516 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:10:06.170235 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-cb99190f-9998-4836-bc42-d2a6852ac516 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:10:06.201381 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:10:06.238248 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb99190f-9998-4836-bc42-d2a6852ac516 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:10:06.239728 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb99190f-9998-4836-bc42-d2a6852ac516 service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:10:06.249131 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1429/2868] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:10:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:10:14.348572 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=52 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:10:16.244986 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=18 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:11:03.205664 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bce6f4-2895-4214-96f4-1122fd6875c7 service nova] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:11:03.207094 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-37bce6f4-2895-4214-96f4-1122fd6875c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:11:03.305704 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bce6f4-2895-4214-96f4-1122fd6875c7 service nova] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:11:03.310556 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805363a-8c40-4cef-82e4-f73360bbd496 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c23a782b9b4144bace1248c3b2e754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:11:03.311150 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805363a-8c40-4cef-82e4-f73360bbd496 service placement] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:11:03.311424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805363a-8c40-4cef-82e4-f73360bbd496 service placement] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:11:03.311689 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805363a-8c40-4cef-82e4-f73360bbd496 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:11:03.313517 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-c805363a-8c40-4cef-82e4-f73360bbd496 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:11:03.325481 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c805363a-8c40-4cef-82e4-f73360bbd496 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:11:03.327032 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c805363a-8c40-4cef-82e4-f73360bbd496 service placement] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:11:03.336621 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1440/2869] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:11:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:11:13.332421 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=44 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:13:04.203652 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99190f-9998-4836-bc42-d2a6852ac516 service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:13:04.205263 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-cb99190f-9998-4836-bc42-d2a6852ac516 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:13:04.281735 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99190f-9998-4836-bc42-d2a6852ac516 service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:13:04.286183 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb661e9-caeb-4b19-8486-6fb9583f81ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:13:04.286700 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb661e9-caeb-4b19-8486-6fb9583f81ca service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:13:04.286976 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb661e9-caeb-4b19-8486-6fb9583f81ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:13:04.287315 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb661e9-caeb-4b19-8486-6fb9583f81ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:13:04.323020 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805363a-8c40-4cef-82e4-f73360bbd496 service placement] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:13:04.333021 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805363a-8c40-4cef-82e4-f73360bbd496 service placement] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:13:04.336597 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3959ab3a-e816-419a-95a6-7fc30fe44328 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:13:04.337222 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959ab3a-e816-419a-95a6-7fc30fe44328 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:13:04.337503 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959ab3a-e816-419a-95a6-7fc30fe44328 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:13:04.337774 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959ab3a-e816-419a-95a6-7fc30fe44328 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:13:04.339594 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3959ab3a-e816-419a-95a6-7fc30fe44328 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:13:04.362678 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb661e9-caeb-4b19-8486-6fb9583f81ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:13:04.364629 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb661e9-caeb-4b19-8486-6fb9583f81ca service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:13:04.369770 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3959ab3a-e816-419a-95a6-7fc30fe44328 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:13:04.371980 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3959ab3a-e816-419a-95a6-7fc30fe44328 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:13:04.391652 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1430/2870] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:13:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:13:04.393978 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1441/2871] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:13:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 09:13:14.387242 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=70 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:13:14.391250 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=15 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:14:04.450894 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce4d4c-f8ed-436e-8f87-8e854480a269 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:14:04.451449 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce4d4c-f8ed-436e-8f87-8e854480a269 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:14:04.451449 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce4d4c-f8ed-436e-8f87-8e854480a269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:14:04.512372 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-2dce4d4c-f8ed-436e-8f87-8e854480a269 None None] MFA Rules not processed for user `7f3422c6f9a041ceba851b71bd15d0df`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 09:14:04.513541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2dce4d4c-f8ed-436e-8f87-8e854480a269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:14:04.580563 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1431/2872] 10.222.0.33 () {58 vars in 967 bytes} [Mon Aug 8 09:14:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 09:14:04.740665 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3959ab3a-e816-419a-95a6-7fc30fe44328 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:14:04.742075 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-3959ab3a-e816-419a-95a6-7fc30fe44328 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:14:04.829659 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3959ab3a-e816-419a-95a6-7fc30fe44328 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:14:04.834536 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e9ed6-939f-41b3-999c-f49c2115ae9f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e69c0e02ad944f2a85fc5318018e5cb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:14:04.835128 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e9ed6-939f-41b3-999c-f49c2115ae9f service swift] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:14:04.835409 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e9ed6-939f-41b3-999c-f49c2115ae9f service swift] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:14:04.835675 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e9ed6-939f-41b3-999c-f49c2115ae9f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:14:04.843420 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e9ed6-939f-41b3-999c-f49c2115ae9f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:14:04.845011 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e9ed6-939f-41b3-999c-f49c2115ae9f service swift] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:14:04.852843 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1442/2873] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 09:14:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 09:14:05.991481 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce4d4c-f8ed-436e-8f87-8e854480a269 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:14:06.001065 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce4d4c-f8ed-436e-8f87-8e854480a269 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:14:06.004864 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00387c81-41db-4ec6-a0a3-5ee0f645c501 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:14:06.005510 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00387c81-41db-4ec6-a0a3-5ee0f645c501 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:14:06.005789 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00387c81-41db-4ec6-a0a3-5ee0f645c501 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:14:06.006058 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00387c81-41db-4ec6-a0a3-5ee0f645c501 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:14:06.007777 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-00387c81-41db-4ec6-a0a3-5ee0f645c501 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:14:06.096994 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00387c81-41db-4ec6-a0a3-5ee0f645c501 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:14:06.098467 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00387c81-41db-4ec6-a0a3-5ee0f645c501 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:14:06.116873 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1432/2874] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:14:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:14:09.627156 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e9ed6-939f-41b3-999c-f49c2115ae9f service swift] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:14:09.638737 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e9ed6-939f-41b3-999c-f49c2115ae9f service swift] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:14:09.644370 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28af07e-a6da-4b67-a838-73fd0ac9fc55 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c23a782b9b4144bace1248c3b2e754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:14:09.644966 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28af07e-a6da-4b67-a838-73fd0ac9fc55 service placement] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:14:09.645271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28af07e-a6da-4b67-a838-73fd0ac9fc55 service placement] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:14:09.645542 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28af07e-a6da-4b67-a838-73fd0ac9fc55 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:14:09.647543 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f28af07e-a6da-4b67-a838-73fd0ac9fc55 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:14:09.660215 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28af07e-a6da-4b67-a838-73fd0ac9fc55 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:14:09.661831 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28af07e-a6da-4b67-a838-73fd0ac9fc55 service placement] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:14:09.672206 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1443/2875] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:14:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:14:12.990110 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00387c81-41db-4ec6-a0a3-5ee0f645c501 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:14:13.001687 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00387c81-41db-4ec6-a0a3-5ee0f645c501 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:14:13.007336 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6e2e8-9253-41ab-99e0-8d3751bf898f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c23a782b9b4144bace1248c3b2e754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:14:13.008029 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6e2e8-9253-41ab-99e0-8d3751bf898f service placement] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:14:13.008311 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6e2e8-9253-41ab-99e0-8d3751bf898f service placement] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:14:13.008580 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6e2e8-9253-41ab-99e0-8d3751bf898f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:14:13.010304 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-2cf6e2e8-9253-41ab-99e0-8d3751bf898f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:14:13.022852 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf6e2e8-9253-41ab-99e0-8d3751bf898f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:14:13.024425 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf6e2e8-9253-41ab-99e0-8d3751bf898f service placement] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:14:13.036738 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1433/2876] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:14:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:14:13.137648 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28af07e-a6da-4b67-a838-73fd0ac9fc55 service placement] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:14:13.139083 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f28af07e-a6da-4b67-a838-73fd0ac9fc55 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:14:13.153887 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28af07e-a6da-4b67-a838-73fd0ac9fc55 service placement] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:14:13.163385 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec6b2c-530e-43b1-ae0f-5932be97f4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:14:13.163385 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec6b2c-530e-43b1-ae0f-5932be97f4e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:14:13.163385 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec6b2c-530e-43b1-ae0f-5932be97f4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:14:13.163385 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec6b2c-530e-43b1-ae0f-5932be97f4e3 admin admin] PATH_INFO: `/v3/users/891658773e7546c78c3213a3f5233fd4` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:14:13.163753 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec6b2c-530e-43b1-ae0f-5932be97f4e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=891658773e7546c78c3213a3f5233fd4)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:14:13.163753 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec6b2c-530e-43b1-ae0f-5932be97f4e3 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:14:13.332184 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8aec6b2c-530e-43b1-ae0f-5932be97f4e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '891658773e7546c78c3213a3f5233fd4'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:14:13.332832 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8aec6b2c-530e-43b1-ae0f-5932be97f4e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '891658773e7546c78c3213a3f5233fd4'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:14:13.333451 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8aec6b2c-530e-43b1-ae0f-5932be97f4e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '891658773e7546c78c3213a3f5233fd4'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:14:13.340196 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8aec6b2c-530e-43b1-ae0f-5932be97f4e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '891658773e7546c78c3213a3f5233fd4'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:14:13.404482 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8aec6b2c-530e-43b1-ae0f-5932be97f4e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '891658773e7546c78c3213a3f5233fd4'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:14:13.424927 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1444/2877] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 09:14:13 2022] DELETE /identity/v3/users/891658773e7546c78c3213a3f5233fd4 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 09:14:13.914343 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6e2e8-9253-41ab-99e0-8d3751bf898f service placement] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:14:13.931981 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6e2e8-9253-41ab-99e0-8d3751bf898f service placement] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:14:13.944618 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=100 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:14:13.989678 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b82612-f9b0-4e3b-82f1-516cc1c77337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:14:13.990310 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b82612-f9b0-4e3b-82f1-516cc1c77337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:14:13.990851 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b82612-f9b0-4e3b-82f1-516cc1c77337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:14:13.990851 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b82612-f9b0-4e3b-82f1-516cc1c77337 admin admin] PATH_INFO: `/v3/projects/0bbf785f7de94ef7ad423568d41af122` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:14:13.991953 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b82612-f9b0-4e3b-82f1-516cc1c77337 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bbf785f7de94ef7ad423568d41af122)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:14:13.993893 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b82612-f9b0-4e3b-82f1-516cc1c77337 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:14:14.107759 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c6b82612-f9b0-4e3b-82f1-516cc1c77337 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bbf785f7de94ef7ad423568d41af122', 'request_id': 'req-c6b82612-f9b0-4e3b-82f1-516cc1c77337'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:14:14.114401 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c6b82612-f9b0-4e3b-82f1-516cc1c77337 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bbf785f7de94ef7ad423568d41af122', 'request_id': 'req-c6b82612-f9b0-4e3b-82f1-516cc1c77337'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:14:14.119831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c6b82612-f9b0-4e3b-82f1-516cc1c77337 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bbf785f7de94ef7ad423568d41af122', 'request_id': 'req-c6b82612-f9b0-4e3b-82f1-516cc1c77337'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:14:14.182071 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c6b82612-f9b0-4e3b-82f1-516cc1c77337 admin admin] The token cache is being invalidate because project 0bbf785f7de94ef7ad423568d41af122 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 09:14:14.182339 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c6b82612-f9b0-4e3b-82f1-516cc1c77337 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:14:14.182982 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c6b82612-f9b0-4e3b-82f1-516cc1c77337 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:14:14.184457 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1434/2878] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 09:14:13 2022] DELETE /identity/v3/projects/0bbf785f7de94ef7ad423568d41af122 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 09:14:14.188510 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec6b2c-530e-43b1-ae0f-5932be97f4e3 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:14:14.189921 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8aec6b2c-530e-43b1-ae0f-5932be97f4e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:14:14.203049 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec6b2c-530e-43b1-ae0f-5932be97f4e3 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:14:14.229466 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:14:14.258540 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40935ad-9187-44b8-83f6-b32b0f032ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:14:14.259162 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40935ad-9187-44b8-83f6-b32b0f032ab3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:14:14.259502 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40935ad-9187-44b8-83f6-b32b0f032ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:14:14.260044 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40935ad-9187-44b8-83f6-b32b0f032ab3 admin admin] PATH_INFO: `/v3/users/54c33c5b6a6e4741a99670befefff1f1` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:14:14.274144 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40935ad-9187-44b8-83f6-b32b0f032ab3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54c33c5b6a6e4741a99670befefff1f1)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:14:14.276125 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40935ad-9187-44b8-83f6-b32b0f032ab3 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:14:14.485934 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f40935ad-9187-44b8-83f6-b32b0f032ab3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54c33c5b6a6e4741a99670befefff1f1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:14:14.486588 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f40935ad-9187-44b8-83f6-b32b0f032ab3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54c33c5b6a6e4741a99670befefff1f1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:14:14.487182 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f40935ad-9187-44b8-83f6-b32b0f032ab3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54c33c5b6a6e4741a99670befefff1f1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:14:14.494095 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f40935ad-9187-44b8-83f6-b32b0f032ab3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54c33c5b6a6e4741a99670befefff1f1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:14:14.558809 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-f40935ad-9187-44b8-83f6-b32b0f032ab3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54c33c5b6a6e4741a99670befefff1f1'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:14:14.577019 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1445/2879] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 09:14:14 2022] DELETE /identity/v3/users/54c33c5b6a6e4741a99670befefff1f1 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 09:14:14.823003 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b82612-f9b0-4e3b-82f1-516cc1c77337 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:14:14.824650 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-c6b82612-f9b0-4e3b-82f1-516cc1c77337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:14:14.843585 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b82612-f9b0-4e3b-82f1-516cc1c77337 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:14:14.909438 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1389adb-5c57-496b-acb4-6fa0bcd63c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:14:14.909998 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1389adb-5c57-496b-acb4-6fa0bcd63c81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:14:14.910276 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1389adb-5c57-496b-acb4-6fa0bcd63c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:14:14.910548 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1389adb-5c57-496b-acb4-6fa0bcd63c81 admin admin] PATH_INFO: `/v3/projects/75dc214097cc42d1b882d5bfcd4d7654` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:14:14.920607 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1389adb-5c57-496b-acb4-6fa0bcd63c81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75dc214097cc42d1b882d5bfcd4d7654)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:14:14.922547 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1389adb-5c57-496b-acb4-6fa0bcd63c81 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:14:15.045157 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:14:15.052349 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c1389adb-5c57-496b-acb4-6fa0bcd63c81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75dc214097cc42d1b882d5bfcd4d7654', 'request_id': 'req-c1389adb-5c57-496b-acb4-6fa0bcd63c81'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:14:15.059115 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c1389adb-5c57-496b-acb4-6fa0bcd63c81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75dc214097cc42d1b882d5bfcd4d7654', 'request_id': 'req-c1389adb-5c57-496b-acb4-6fa0bcd63c81'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:14:15.064391 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c1389adb-5c57-496b-acb4-6fa0bcd63c81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75dc214097cc42d1b882d5bfcd4d7654', 'request_id': 'req-c1389adb-5c57-496b-acb4-6fa0bcd63c81'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:14:15.141438 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c1389adb-5c57-496b-acb4-6fa0bcd63c81 admin admin] The token cache is being invalidate because project 75dc214097cc42d1b882d5bfcd4d7654 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 09:14:15.141726 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c1389adb-5c57-496b-acb4-6fa0bcd63c81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:14:15.142618 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-c1389adb-5c57-496b-acb4-6fa0bcd63c81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:14:15.144251 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1435/2880] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 09:14:14 2022] DELETE /identity/v3/projects/75dc214097cc42d1b882d5bfcd4d7654 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 09:14:15.149444 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db64d38-4aa4-4777-be41-7d19da8a81c1 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:14:15.149734 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db64d38-4aa4-4777-be41-7d19da8a81c1 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:14:15.150013 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db64d38-4aa4-4777-be41-7d19da8a81c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:14:15.208010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-5db64d38-4aa4-4777-be41-7d19da8a81c1 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 09:14:15.209128 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-5db64d38-4aa4-4777-be41-7d19da8a81c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:14:15.265992 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1446/2881] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 09:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 09:14:15.274499 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeed64b-23eb-48a9-9160-96f1fd8920fb None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:14:15.274784 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeed64b-23eb-48a9-9160-96f1fd8920fb None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:14:15.275057 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeed64b-23eb-48a9-9160-96f1fd8920fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:14:15.293905 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-ebeed64b-23eb-48a9-9160-96f1fd8920fb None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 09:14:15.295093 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-ebeed64b-23eb-48a9-9160-96f1fd8920fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:14:15.303551 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1436/2882] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 09:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 09:14:15.307847 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db64d38-4aa4-4777-be41-7d19da8a81c1 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:14:15.318051 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db64d38-4aa4-4777-be41-7d19da8a81c1 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:14:15.322613 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e0014-e0ec-41e2-bec1-c5c8c2cc8076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:14:15.323207 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e0014-e0ec-41e2-bec1-c5c8c2cc8076 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:14:15.323491 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e0014-e0ec-41e2-bec1-c5c8c2cc8076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:14:15.323764 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e0014-e0ec-41e2-bec1-c5c8c2cc8076 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:14:15.324410 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e0014-e0ec-41e2-bec1-c5c8c2cc8076 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 09:14:15.324712 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e0014-e0ec-41e2-bec1-c5c8c2cc8076 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:14:15.326348 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e0014-e0ec-41e2-bec1-c5c8c2cc8076 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:14:15.327825 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:14:15.338701 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1447/2883] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 09:14:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 09:14:15.342862 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeed64b-23eb-48a9-9160-96f1fd8920fb None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:14:15.352485 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeed64b-23eb-48a9-9160-96f1fd8920fb None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:14:15.357050 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bfa71-7141-43d2-87a4-fac529bb1811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:14:15.357849 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bfa71-7141-43d2-87a4-fac529bb1811 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:14:15.358303 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bfa71-7141-43d2-87a4-fac529bb1811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:14:15.358581 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bfa71-7141-43d2-87a4-fac529bb1811 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:14:15.359210 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65bfa71-7141-43d2-87a4-fac529bb1811 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:14:15.361107 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65bfa71-7141-43d2-87a4-fac529bb1811 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:14:15.457804 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1437/2884] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 09:14:15 2022] POST /identity/v3/projects => generated 371 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 09:14:15.461974 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e0014-e0ec-41e2-bec1-c5c8c2cc8076 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:14:15.472870 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e0014-e0ec-41e2-bec1-c5c8c2cc8076 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:14:15.528557 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943c58e-a3fe-4531-b631-b87d9620a861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:14:15.529186 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943c58e-a3fe-4531-b631-b87d9620a861 admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:14:15.529473 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943c58e-a3fe-4531-b631-b87d9620a861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:14:15.529748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943c58e-a3fe-4531-b631-b87d9620a861 admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:14:15.530374 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8943c58e-a3fe-4531-b631-b87d9620a861 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:14:15.532292 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8943c58e-a3fe-4531-b631-b87d9620a861 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:14:15.639290 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1448/2885] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 09:14:15 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 09:14:15.643747 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bfa71-7141-43d2-87a4-fac529bb1811 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:14:15.654023 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bfa71-7141-43d2-87a4-fac529bb1811 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:14:15.658577 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a459d6-4c0c-4a54-b81e-39aa9c3a9ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:14:15.659172 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a459d6-4c0c-4a54-b81e-39aa9c3a9ecb admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:14:15.659938 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a459d6-4c0c-4a54-b81e-39aa9c3a9ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:14:15.660253 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a459d6-4c0c-4a54-b81e-39aa9c3a9ecb admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:14:15.660921 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a459d6-4c0c-4a54-b81e-39aa9c3a9ecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:14:15.662531 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a459d6-4c0c-4a54-b81e-39aa9c3a9ecb admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:14:15.672178 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1438/2886] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 09:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 09:14:15.679467 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943c58e-a3fe-4531-b631-b87d9620a861 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:14:15.689411 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943c58e-a3fe-4531-b631-b87d9620a861 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:14:15.694108 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fab14a-9e82-471f-8260-9ded668cbc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:14:15.694688 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fab14a-9e82-471f-8260-9ded668cbc1c admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:14:15.694964 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fab14a-9e82-471f-8260-9ded668cbc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:14:15.695304 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fab14a-9e82-471f-8260-9ded668cbc1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:14:15.695937 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fab14a-9e82-471f-8260-9ded668cbc1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:14:15.697555 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fab14a-9e82-471f-8260-9ded668cbc1c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:14:15.707592 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1449/2887] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 09:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 09:14:15.711531 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a459d6-4c0c-4a54-b81e-39aa9c3a9ecb admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:14:15.721271 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a459d6-4c0c-4a54-b81e-39aa9c3a9ecb admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:14:15.726487 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d19248-2820-47ee-94c8-93d2629b255c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:14:15.727120 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d19248-2820-47ee-94c8-93d2629b255c admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:14:15.727400 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d19248-2820-47ee-94c8-93d2629b255c admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:14:15.727675 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d19248-2820-47ee-94c8-93d2629b255c admin admin] PATH_INFO: `/v3/projects/b7f5daa7b9e84f7e866fc990001b8666/users/2754079f211f4a30a60cd38911f3305b/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:14:15.741914 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d19248-2820-47ee-94c8-93d2629b255c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7f5daa7b9e84f7e866fc990001b8666, user_id=2754079f211f4a30a60cd38911f3305b, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:14:15.744541 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d19248-2820-47ee-94c8-93d2629b255c admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:14:15.815077 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1439/2888] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 09:14:15 2022] PUT /identity/v3/projects/b7f5daa7b9e84f7e866fc990001b8666/users/2754079f211f4a30a60cd38911f3305b/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 09:14:15.820437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa148fb-d485-4d31-918e-f4c63ad891f2 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:14:15.820727 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa148fb-d485-4d31-918e-f4c63ad891f2 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:14:15.820999 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa148fb-d485-4d31-918e-f4c63ad891f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:14:15.840259 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-caa148fb-d485-4d31-918e-f4c63ad891f2 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 09:14:15.841476 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-caa148fb-d485-4d31-918e-f4c63ad891f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:14:15.899241 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1450/2889] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 09:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 09:14:15.909030 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d19248-2820-47ee-94c8-93d2629b255c admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:14:15.911360 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-f9d19248-2820-47ee-94c8-93d2629b255c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:14:15.965604 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d19248-2820-47ee-94c8-93d2629b255c admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:14:15.969174 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f89ee2-cb82-476f-ad4f-6bd114a12c3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:14:15.969760 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f89ee2-cb82-476f-ad4f-6bd114a12c3a service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:14:15.970035 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f89ee2-cb82-476f-ad4f-6bd114a12c3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:14:15.970306 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f89ee2-cb82-476f-ad4f-6bd114a12c3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:14:15.979741 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f89ee2-cb82-476f-ad4f-6bd114a12c3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:14:15.981080 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f89ee2-cb82-476f-ad4f-6bd114a12c3a service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:14:15.990306 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1440/2890] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:14:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:14:21.210804 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780b878-8b8e-4875-ae9e-29b0bb13f620 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:14:21.211301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780b878-8b8e-4875-ae9e-29b0bb13f620 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:14:21.211385 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780b878-8b8e-4875-ae9e-29b0bb13f620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:14:21.245366 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-0780b878-8b8e-4875-ae9e-29b0bb13f620 None None] MFA Rules not processed for user `2754079f211f4a30a60cd38911f3305b`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 09:14:21.246674 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0780b878-8b8e-4875-ae9e-29b0bb13f620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:14:21.317044 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1451/2891] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 09:14:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 09:14:21.321954 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f89ee2-cb82-476f-ad4f-6bd114a12c3a service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:14:21.339353 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f89ee2-cb82-476f-ad4f-6bd114a12c3a service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:14:21.344103 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe2050-8e4a-4e0b-9fbe-08c97662a877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:14:21.344705 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe2050-8e4a-4e0b-9fbe-08c97662a877 admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:14:21.344984 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe2050-8e4a-4e0b-9fbe-08c97662a877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:14:21.345301 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe2050-8e4a-4e0b-9fbe-08c97662a877 admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:14:21.345931 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe2050-8e4a-4e0b-9fbe-08c97662a877 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:14:21.348938 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe2050-8e4a-4e0b-9fbe-08c97662a877 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:14:21.462020 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1441/2892] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 09:14:21 2022] POST /identity/v3/projects => generated 373 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 09:14:21.466220 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780b878-8b8e-4875-ae9e-29b0bb13f620 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:14:21.471667 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:14:21.478048 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780b878-8b8e-4875-ae9e-29b0bb13f620 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:14:21.535010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e55d36-dea4-41a1-a6b4-fc3f97bdde9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:14:21.535754 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e55d36-dea4-41a1-a6b4-fc3f97bdde9f admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:14:21.536037 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e55d36-dea4-41a1-a6b4-fc3f97bdde9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:14:21.536308 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e55d36-dea4-41a1-a6b4-fc3f97bdde9f admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:14:21.536963 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e55d36-dea4-41a1-a6b4-fc3f97bdde9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:14:21.538946 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e55d36-dea4-41a1-a6b4-fc3f97bdde9f admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:14:21.629979 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1452/2893] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 09:14:21 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 09:14:21.631164 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe2050-8e4a-4e0b-9fbe-08c97662a877 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:14:21.641705 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe2050-8e4a-4e0b-9fbe-08c97662a877 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:14:21.646498 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc78e34-75e9-4713-bf58-17684c9a3abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:14:21.647285 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc78e34-75e9-4713-bf58-17684c9a3abe admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:14:21.647464 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc78e34-75e9-4713-bf58-17684c9a3abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:14:21.647689 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc78e34-75e9-4713-bf58-17684c9a3abe admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:14:21.648320 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc78e34-75e9-4713-bf58-17684c9a3abe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:14:21.650003 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc78e34-75e9-4713-bf58-17684c9a3abe admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:14:21.661079 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1442/2894] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 09:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 09:14:21.665395 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e55d36-dea4-41a1-a6b4-fc3f97bdde9f admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:14:21.676581 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e55d36-dea4-41a1-a6b4-fc3f97bdde9f admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:14:21.681164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78a1ce-3ebf-476b-874d-a5b611194591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:14:21.681829 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78a1ce-3ebf-476b-874d-a5b611194591 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:14:21.682109 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78a1ce-3ebf-476b-874d-a5b611194591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:14:21.682400 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78a1ce-3ebf-476b-874d-a5b611194591 admin admin] PATH_INFO: `/v3/projects/2e474883ca6b41bbaa6cfce30bf0d80b/users/9f08a18c1dbd430692d65f66bbbe5f8e/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:14:21.697874 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78a1ce-3ebf-476b-874d-a5b611194591 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e474883ca6b41bbaa6cfce30bf0d80b, user_id=9f08a18c1dbd430692d65f66bbbe5f8e, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:14:21.700525 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78a1ce-3ebf-476b-874d-a5b611194591 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:14:21.782920 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1453/2895] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 09:14:21 2022] PUT /identity/v3/projects/2e474883ca6b41bbaa6cfce30bf0d80b/users/9f08a18c1dbd430692d65f66bbbe5f8e/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 09:14:21.789333 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc78e34-75e9-4713-bf58-17684c9a3abe admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:14:21.799939 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc78e34-75e9-4713-bf58-17684c9a3abe admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:14:21.820851 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:14:21.857751 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7537bd-13e1-4051-8d85-8445ccfbaf0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:14:21.858435 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7537bd-13e1-4051-8d85-8445ccfbaf0b admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:14:21.858717 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7537bd-13e1-4051-8d85-8445ccfbaf0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:14:21.858998 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7537bd-13e1-4051-8d85-8445ccfbaf0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:14:21.859679 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7537bd-13e1-4051-8d85-8445ccfbaf0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:14:21.861286 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7537bd-13e1-4051-8d85-8445ccfbaf0b admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:14:21.872454 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1443/2896] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 09:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 09:14:21.876748 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78a1ce-3ebf-476b-874d-a5b611194591 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:14:21.888066 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78a1ce-3ebf-476b-874d-a5b611194591 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:14:21.896654 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063fbd9-13f2-46f4-891f-1efd61e94390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:14:21.897353 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063fbd9-13f2-46f4-891f-1efd61e94390 admin admin] REQUEST_METHOD: `PUT` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:14:21.897634 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063fbd9-13f2-46f4-891f-1efd61e94390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:14:21.897903 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063fbd9-13f2-46f4-891f-1efd61e94390 admin admin] PATH_INFO: `/v3/projects/2e474883ca6b41bbaa6cfce30bf0d80b/users/9f08a18c1dbd430692d65f66bbbe5f8e/roles/c1e514a4b5424f959b069f36cbfd8147` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:14:21.899869 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7063fbd9-13f2-46f4-891f-1efd61e94390 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e474883ca6b41bbaa6cfce30bf0d80b, user_id=9f08a18c1dbd430692d65f66bbbe5f8e, role_id=c1e514a4b5424f959b069f36cbfd8147)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:14:21.902521 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7063fbd9-13f2-46f4-891f-1efd61e94390 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:14:21.914318 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1454/2897] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 09:14:21 2022] PUT /identity/v3/projects/2e474883ca6b41bbaa6cfce30bf0d80b/users/9f08a18c1dbd430692d65f66bbbe5f8e/roles/c1e514a4b5424f959b069f36cbfd8147 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 09:14:27.567503 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f77cf-9e71-4f32-981e-74e4619b8871 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:14:27.567943 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f77cf-9e71-4f32-981e-74e4619b8871 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:14:27.568158 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f77cf-9e71-4f32-981e-74e4619b8871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:14:27.598578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-438f77cf-9e71-4f32-981e-74e4619b8871 None None] MFA Rules not processed for user `9f08a18c1dbd430692d65f66bbbe5f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 09:14:27.599766 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-438f77cf-9e71-4f32-981e-74e4619b8871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:14:27.672667 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1444/2898] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 09:14:27 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 09:14:27.681398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063fbd9-13f2-46f4-891f-1efd61e94390 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:14:27.683109 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7063fbd9-13f2-46f4-891f-1efd61e94390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:14:27.762790 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063fbd9-13f2-46f4-891f-1efd61e94390 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:14:27.768559 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06677dea-43c3-4872-a357-386612adf772 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:14:27.769280 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06677dea-43c3-4872-a357-386612adf772 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:14:27.769550 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06677dea-43c3-4872-a357-386612adf772 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:14:27.769814 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06677dea-43c3-4872-a357-386612adf772 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:14:27.817472 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:14:27.828443 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06677dea-43c3-4872-a357-386612adf772 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:14:27.830240 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06677dea-43c3-4872-a357-386612adf772 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:14:27.841338 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1455/2899] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:14:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:14:30.383487 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fd966-3b72-4cd5-9c07-baf7a88df818 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:14:30.383919 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fd966-3b72-4cd5-9c07-baf7a88df818 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:14:30.384023 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fd966-3b72-4cd5-9c07-baf7a88df818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:14:30.403530 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-e32fd966-3b72-4cd5-9c07-baf7a88df818 None None] MFA Rules not processed for user `2754079f211f4a30a60cd38911f3305b`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 09:14:30.404714 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e32fd966-3b72-4cd5-9c07-baf7a88df818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:14:30.416524 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1445/2900] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 09:14:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 09:14:37.930876 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=4 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:14:40.398010 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=55 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:15:05.880205 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06677dea-43c3-4872-a357-386612adf772 service nova] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:15:05.882516 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-06677dea-43c3-4872-a357-386612adf772 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:15:05.897663 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06677dea-43c3-4872-a357-386612adf772 service nova] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:15:05.903294 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecad8be-0a26-4c77-8c53-11545073f2ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:15:05.903889 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecad8be-0a26-4c77-8c53-11545073f2ab service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:15:05.904162 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecad8be-0a26-4c77-8c53-11545073f2ab service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:15:05.904437 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecad8be-0a26-4c77-8c53-11545073f2ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:15:05.906135 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-cecad8be-0a26-4c77-8c53-11545073f2ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:15:05.971981 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecad8be-0a26-4c77-8c53-11545073f2ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:15:05.973586 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecad8be-0a26-4c77-8c53-11545073f2ab service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:15:05.991846 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1456/2901] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:15:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:15:15.989249 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=36 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:15:37.904744 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32fd966-3b72-4cd5-9c07-baf7a88df818 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:15:37.968225 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32fd966-3b72-4cd5-9c07-baf7a88df818 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:15:37.975085 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967717c2-3461-422f-a615-640533c3cfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:15:37.975775 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967717c2-3461-422f-a615-640533c3cfc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:15:37.976592 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967717c2-3461-422f-a615-640533c3cfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:15:37.977008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967717c2-3461-422f-a615-640533c3cfc1 admin admin] PATH_INFO: `/v3/users/2754079f211f4a30a60cd38911f3305b` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:15:37.978173 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967717c2-3461-422f-a615-640533c3cfc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2754079f211f4a30a60cd38911f3305b)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:15:37.980081 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967717c2-3461-422f-a615-640533c3cfc1 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:15:38.139076 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-967717c2-3461-422f-a615-640533c3cfc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2754079f211f4a30a60cd38911f3305b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:15:38.139650 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-967717c2-3461-422f-a615-640533c3cfc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2754079f211f4a30a60cd38911f3305b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:15:38.140270 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-967717c2-3461-422f-a615-640533c3cfc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2754079f211f4a30a60cd38911f3305b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:15:38.147579 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-967717c2-3461-422f-a615-640533c3cfc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2754079f211f4a30a60cd38911f3305b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:15:38.228603 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-967717c2-3461-422f-a615-640533c3cfc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2754079f211f4a30a60cd38911f3305b'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:15:38.247962 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1446/2902] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 09:15:37 2022] DELETE /identity/v3/users/2754079f211f4a30a60cd38911f3305b => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 09:15:38.500195 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecad8be-0a26-4c77-8c53-11545073f2ab service nova] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:15:38.512940 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecad8be-0a26-4c77-8c53-11545073f2ab service nova] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:15:38.570769 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c3bd8-a92c-4137-a37b-d524b8ff26fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:15:38.571398 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c3bd8-a92c-4137-a37b-d524b8ff26fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:15:38.571677 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c3bd8-a92c-4137-a37b-d524b8ff26fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:15:38.573313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c3bd8-a92c-4137-a37b-d524b8ff26fb admin admin] PATH_INFO: `/v3/projects/b7f5daa7b9e84f7e866fc990001b8666` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:15:38.573313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665c3bd8-a92c-4137-a37b-d524b8ff26fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7f5daa7b9e84f7e866fc990001b8666)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:15:38.575046 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665c3bd8-a92c-4137-a37b-d524b8ff26fb admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:15:38.668397 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-665c3bd8-a92c-4137-a37b-d524b8ff26fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7f5daa7b9e84f7e866fc990001b8666', 'request_id': 'req-665c3bd8-a92c-4137-a37b-d524b8ff26fb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:15:38.752302 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-665c3bd8-a92c-4137-a37b-d524b8ff26fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7f5daa7b9e84f7e866fc990001b8666', 'request_id': 'req-665c3bd8-a92c-4137-a37b-d524b8ff26fb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:15:38.759858 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-665c3bd8-a92c-4137-a37b-d524b8ff26fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7f5daa7b9e84f7e866fc990001b8666', 'request_id': 'req-665c3bd8-a92c-4137-a37b-d524b8ff26fb'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:15:38.770553 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-665c3bd8-a92c-4137-a37b-d524b8ff26fb admin admin] The token cache is being invalidate because project b7f5daa7b9e84f7e866fc990001b8666 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 09:15:38.770832 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-665c3bd8-a92c-4137-a37b-d524b8ff26fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:15:38.771446 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-665c3bd8-a92c-4137-a37b-d524b8ff26fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:15:38.772951 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1457/2903] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 09:15:38 2022] DELETE /identity/v3/projects/b7f5daa7b9e84f7e866fc990001b8666 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 09:15:38.777811 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967717c2-3461-422f-a615-640533c3cfc1 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:15:38.779256 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-967717c2-3461-422f-a615-640533c3cfc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:15:38.792244 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967717c2-3461-422f-a615-640533c3cfc1 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:15:38.847307 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db6f7e-ec85-4e06-a32e-632dcab6bc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:15:38.847926 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db6f7e-ec85-4e06-a32e-632dcab6bc0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:15:38.848259 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db6f7e-ec85-4e06-a32e-632dcab6bc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:15:38.848534 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db6f7e-ec85-4e06-a32e-632dcab6bc0c admin admin] PATH_INFO: `/v3/users/9f08a18c1dbd430692d65f66bbbe5f8e` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:15:38.849678 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db6f7e-ec85-4e06-a32e-632dcab6bc0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f08a18c1dbd430692d65f66bbbe5f8e)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:15:38.851831 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db6f7e-ec85-4e06-a32e-632dcab6bc0c admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:15:38.872816 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:15:39.032101 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-37db6f7e-ec85-4e06-a32e-632dcab6bc0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f08a18c1dbd430692d65f66bbbe5f8e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:15:39.032702 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-37db6f7e-ec85-4e06-a32e-632dcab6bc0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f08a18c1dbd430692d65f66bbbe5f8e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:15:39.033308 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-37db6f7e-ec85-4e06-a32e-632dcab6bc0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f08a18c1dbd430692d65f66bbbe5f8e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:15:39.040468 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-37db6f7e-ec85-4e06-a32e-632dcab6bc0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f08a18c1dbd430692d65f66bbbe5f8e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:15:39.106759 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-37db6f7e-ec85-4e06-a32e-632dcab6bc0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f08a18c1dbd430692d65f66bbbe5f8e'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:15:39.124405 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1447/2904] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 09:15:38 2022] DELETE /identity/v3/users/9f08a18c1dbd430692d65f66bbbe5f8e => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 09:15:39.360571 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c3bd8-a92c-4137-a37b-d524b8ff26fb admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:15:39.362070 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-665c3bd8-a92c-4137-a37b-d524b8ff26fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:15:39.376798 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c3bd8-a92c-4137-a37b-d524b8ff26fb admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:15:39.442820 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3602a-c79a-4de9-b703-a592f7312524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:15:39.443498 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3602a-c79a-4de9-b703-a592f7312524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:15:39.443761 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3602a-c79a-4de9-b703-a592f7312524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:15:39.444028 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3602a-c79a-4de9-b703-a592f7312524 admin admin] PATH_INFO: `/v3/projects/2e474883ca6b41bbaa6cfce30bf0d80b` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:15:39.445193 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf3602a-c79a-4de9-b703-a592f7312524 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e474883ca6b41bbaa6cfce30bf0d80b)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:15:39.447263 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf3602a-c79a-4de9-b703-a592f7312524 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:15:39.546821 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:15:39.559560 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8cf3602a-c79a-4de9-b703-a592f7312524 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e474883ca6b41bbaa6cfce30bf0d80b', 'request_id': 'req-8cf3602a-c79a-4de9-b703-a592f7312524'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:15:39.668391 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8cf3602a-c79a-4de9-b703-a592f7312524 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e474883ca6b41bbaa6cfce30bf0d80b', 'request_id': 'req-8cf3602a-c79a-4de9-b703-a592f7312524'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:15:39.674847 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8cf3602a-c79a-4de9-b703-a592f7312524 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e474883ca6b41bbaa6cfce30bf0d80b', 'request_id': 'req-8cf3602a-c79a-4de9-b703-a592f7312524'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:15:39.685888 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8cf3602a-c79a-4de9-b703-a592f7312524 admin admin] The token cache is being invalidate because project 2e474883ca6b41bbaa6cfce30bf0d80b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80261) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 09:15:39.686192 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8cf3602a-c79a-4de9-b703-a592f7312524 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:15:39.686808 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-8cf3602a-c79a-4de9-b703-a592f7312524 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:15:39.688282 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1458/2905] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 09:15:39 2022] DELETE /identity/v3/projects/2e474883ca6b41bbaa6cfce30bf0d80b => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 09:15:49.120749 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:15:49.725849 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:16:07.226855 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db6f7e-ec85-4e06-a32e-632dcab6bc0c admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:16:07.228902 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-37db6f7e-ec85-4e06-a32e-632dcab6bc0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:16:07.297569 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db6f7e-ec85-4e06-a32e-632dcab6bc0c admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:16:07.356468 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490fb79d-4264-4f77-953a-b964dcb6a389 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c23a782b9b4144bace1248c3b2e754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:16:07.357112 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fb79d-4264-4f77-953a-b964dcb6a389 service placement] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:16:07.357516 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fb79d-4264-4f77-953a-b964dcb6a389 service placement] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:16:07.357783 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fb79d-4264-4f77-953a-b964dcb6a389 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:16:07.359493 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-490fb79d-4264-4f77-953a-b964dcb6a389 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:16:07.370633 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490fb79d-4264-4f77-953a-b964dcb6a389 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:16:07.372183 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490fb79d-4264-4f77-953a-b964dcb6a389 service placement] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:16:07.382347 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1448/2906] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:16:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:16:17.377807 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=58 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:16:27.862747 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3602a-c79a-4de9-b703-a592f7312524 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:16:27.864164 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-8cf3602a-c79a-4de9-b703-a592f7312524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:16:27.930649 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3602a-c79a-4de9-b703-a592f7312524 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:16:27.934783 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7175ea5-0b03-4487-8804-9a19abc84ca5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:16:27.935384 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7175ea5-0b03-4487-8804-9a19abc84ca5 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:16:27.935660 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7175ea5-0b03-4487-8804-9a19abc84ca5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:16:27.935927 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7175ea5-0b03-4487-8804-9a19abc84ca5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:16:27.937667 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-d7175ea5-0b03-4487-8804-9a19abc84ca5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:16:27.995463 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7175ea5-0b03-4487-8804-9a19abc84ca5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:16:27.996806 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7175ea5-0b03-4487-8804-9a19abc84ca5 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:16:28.005498 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1459/2907] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:16:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:16:38.001575 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=52 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:18:04.677223 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490fb79d-4264-4f77-953a-b964dcb6a389 service placement] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:18:04.678559 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-490fb79d-4264-4f77-953a-b964dcb6a389 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:18:04.732377 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490fb79d-4264-4f77-953a-b964dcb6a389 service placement] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:18:04.736414 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f172f09-193d-4a1d-950e-fd8b58060a39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:18:04.737008 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f172f09-193d-4a1d-950e-fd8b58060a39 service neutron] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:18:04.737316 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f172f09-193d-4a1d-950e-fd8b58060a39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:18:04.737585 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f172f09-193d-4a1d-950e-fd8b58060a39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:18:04.739264 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0f172f09-193d-4a1d-950e-fd8b58060a39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:18:04.808598 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f172f09-193d-4a1d-950e-fd8b58060a39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:18:04.810197 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f172f09-193d-4a1d-950e-fd8b58060a39 service neutron] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:18:04.821737 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1449/2908] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:18:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:18:05.929548 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7175ea5-0b03-4487-8804-9a19abc84ca5 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:18:05.938649 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7175ea5-0b03-4487-8804-9a19abc84ca5 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:18:05.942774 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289b9cb-e83e-4225-8182-25d479709a35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:18:05.943360 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289b9cb-e83e-4225-8182-25d479709a35 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:18:05.943632 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289b9cb-e83e-4225-8182-25d479709a35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:18:05.943897 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289b9cb-e83e-4225-8182-25d479709a35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:18:05.945593 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-4289b9cb-e83e-4225-8182-25d479709a35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:18:05.959213 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289b9cb-e83e-4225-8182-25d479709a35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:18:05.960846 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289b9cb-e83e-4225-8182-25d479709a35 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:18:05.972495 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1460/2909] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:18:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:18:06.445117 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f172f09-193d-4a1d-950e-fd8b58060a39 service neutron] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:18:06.446578 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-0f172f09-193d-4a1d-950e-fd8b58060a39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:18:06.458477 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f172f09-193d-4a1d-950e-fd8b58060a39 service neutron] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:18:06.461972 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4f506-a981-45ae-a23a-3cb0a1ba4c4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:18:06.462561 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4f506-a981-45ae-a23a-3cb0a1ba4c4b service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:18:06.462836 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4f506-a981-45ae-a23a-3cb0a1ba4c4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:18:06.463107 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4f506-a981-45ae-a23a-3cb0a1ba4c4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:18:06.464717 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-97c4f506-a981-45ae-a23a-3cb0a1ba4c4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:18:06.477188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application [None req-97c4f506-a981-45ae-a23a-3cb0a1ba4c4b service cinder] Could not find project: 0bbf785f7de94ef7ad423568d41af122.: keystone.exception.ProjectNotFound: Could not find project: 0bbf785f7de94ef7ad423568d41af122. Aug 08 09:18:06.477188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 09:18:06.477188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 09:18:06.477188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 09:18:06.477188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 09:18:06.477188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 09:18:06.477188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 09:18:06.477188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 09:18:06.477188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 09:18:06.477188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 09:18:06.477188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 09:18:06.477188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 09:18:06.477188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Aug 08 09:18:06.477188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Aug 08 09:18:06.477188 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Aug 08 09:18:06.478233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Aug 08 09:18:06.478233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Aug 08 09:18:06.478233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Aug 08 09:18:06.478233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 09:18:06.478233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 09:18:06.478233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Aug 08 09:18:06.478233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Aug 08 09:18:06.478233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 09:18:06.478233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 09:18:06.478233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 09:18:06.478233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 09:18:06.478233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 09:18:06.478233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 09:18:06.478233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 09:18:06.478233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 09:18:06.478233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 09:18:06.478233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 09:18:06.479170 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 09:18:06.479170 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 09:18:06.479170 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 09:18:06.479170 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 09:18:06.479170 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Aug 08 09:18:06.479170 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Aug 08 09:18:06.479170 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Aug 08 09:18:06.479170 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self._validate_token_resources() Aug 08 09:18:06.479170 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Aug 08 09:18:06.479170 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Aug 08 09:18:06.479170 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Aug 08 09:18:06.479170 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Aug 08 09:18:06.479170 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 09:18:06.479170 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 09:18:06.479170 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 09:18:06.479170 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 09:18:06.479170 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 09:18:06.480054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 09:18:06.480054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 09:18:06.480054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application with Lock( Aug 08 09:18:06.480054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 09:18:06.480054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._enter() Aug 08 09:18:06.480054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 09:18:06.480054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 09:18:06.480054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 09:18:06.480054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.creator() Aug 08 09:18:06.480054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 09:18:06.480054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application created_value = creator( Aug 08 09:18:06.480054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 09:18:06.480054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 09:18:06.480054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 09:18:06.480054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 09:18:06.480054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 09:18:06.480054 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 09:18:06.480950 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 0bbf785f7de94ef7ad423568d41af122. Aug 08 09:18:06.480950 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: ERROR keystone.server.flask.application Aug 08 09:18:06.480950 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1450/2910] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:18:06 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 34 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 08 09:18:15.967609 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=6 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:18:16.468045 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=58 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:20:06.587584 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289b9cb-e83e-4225-8182-25d479709a35 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:20:06.607035 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289b9cb-e83e-4225-8182-25d479709a35 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:20:06.611107 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44875f-8b30-4c8b-8ba9-36f3ef7f5477 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:20:06.612841 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44875f-8b30-4c8b-8ba9-36f3ef7f5477 service cinder] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:20:06.612841 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44875f-8b30-4c8b-8ba9-36f3ef7f5477 service cinder] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:20:06.612841 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44875f-8b30-4c8b-8ba9-36f3ef7f5477 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:20:06.614537 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7c44875f-8b30-4c8b-8ba9-36f3ef7f5477 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:20:06.707412 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c44875f-8b30-4c8b-8ba9-36f3ef7f5477 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:20:06.708906 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c44875f-8b30-4c8b-8ba9-36f3ef7f5477 service cinder] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:20:06.717821 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1461/2911] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:20:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:20:16.713701 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=48 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:21:15.484493 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4f506-a981-45ae-a23a-3cb0a1ba4c4b service cinder] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:21:15.486835 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-97c4f506-a981-45ae-a23a-3cb0a1ba4c4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:21:15.550428 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4f506-a981-45ae-a23a-3cb0a1ba4c4b service cinder] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:21:15.555424 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7329a48-619e-4187-ad65-80fb617b724f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c23a782b9b4144bace1248c3b2e754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:21:15.556016 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7329a48-619e-4187-ad65-80fb617b724f service placement] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:21:15.556290 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7329a48-619e-4187-ad65-80fb617b724f service placement] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:21:15.556553 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7329a48-619e-4187-ad65-80fb617b724f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:21:15.564969 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7329a48-619e-4187-ad65-80fb617b724f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:21:15.566550 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7329a48-619e-4187-ad65-80fb617b724f service placement] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:21:15.578613 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1451/2912] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:21:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:21:16.298068 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44875f-8b30-4c8b-8ba9-36f3ef7f5477 service cinder] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:21:16.299544 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-7c44875f-8b30-4c8b-8ba9-36f3ef7f5477 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:21:16.406026 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44875f-8b30-4c8b-8ba9-36f3ef7f5477 service cinder] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:21:16.411372 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175d697-f215-4fd2-851c-5af6cae47f6f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '627171bf2a2e4a21ba6f800879f8c665', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:21:16.412149 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175d697-f215-4fd2-851c-5af6cae47f6f service glance] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:21:16.412431 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175d697-f215-4fd2-851c-5af6cae47f6f service glance] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:21:16.412860 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175d697-f215-4fd2-851c-5af6cae47f6f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:21:16.414614 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-0175d697-f215-4fd2-851c-5af6cae47f6f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:21:16.429313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application [None req-0175d697-f215-4fd2-851c-5af6cae47f6f service glance] Could not find project: b7f5daa7b9e84f7e866fc990001b8666.: keystone.exception.ProjectNotFound: Could not find project: b7f5daa7b9e84f7e866fc990001b8666. Aug 08 09:21:16.429313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 09:21:16.429313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 09:21:16.429313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 09:21:16.429313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 09:21:16.429313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 09:21:16.429313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 09:21:16.429313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 09:21:16.429313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 09:21:16.429313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 09:21:16.429313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 09:21:16.429313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 09:21:16.429313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Aug 08 09:21:16.429313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Aug 08 09:21:16.429313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Aug 08 09:21:16.430260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Aug 08 09:21:16.430260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Aug 08 09:21:16.430260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Aug 08 09:21:16.430260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 09:21:16.430260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 09:21:16.430260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Aug 08 09:21:16.430260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Aug 08 09:21:16.430260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 09:21:16.430260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 09:21:16.430260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 09:21:16.430260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 09:21:16.430260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 09:21:16.430260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 09:21:16.430260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 09:21:16.430260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 09:21:16.430260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 09:21:16.430260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 09:21:16.431131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 09:21:16.431131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 09:21:16.431131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 09:21:16.431131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 09:21:16.431131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Aug 08 09:21:16.431131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Aug 08 09:21:16.431131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Aug 08 09:21:16.431131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self._validate_token_resources() Aug 08 09:21:16.431131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Aug 08 09:21:16.431131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Aug 08 09:21:16.431131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Aug 08 09:21:16.431131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Aug 08 09:21:16.431131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 09:21:16.431131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 09:21:16.431131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 09:21:16.431131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 09:21:16.431131 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 09:21:16.432015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 09:21:16.432015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 09:21:16.432015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application with Lock( Aug 08 09:21:16.432015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 09:21:16.432015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._enter() Aug 08 09:21:16.432015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 09:21:16.432015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 09:21:16.432015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 09:21:16.432015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.creator() Aug 08 09:21:16.432015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 09:21:16.432015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application created_value = creator( Aug 08 09:21:16.432015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 09:21:16.432015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 09:21:16.432015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 09:21:16.432015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 09:21:16.432015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 09:21:16.432015 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 09:21:16.432829 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: b7f5daa7b9e84f7e866fc990001b8666. Aug 08 09:21:16.432829 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: ERROR keystone.server.flask.application Aug 08 09:21:16.432829 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1462/2913] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:21:16 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 133 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 08 09:21:22.162265 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e1d62-b7a4-4913-8a38-f288d3bffa91 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:21:22.162789 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e1d62-b7a4-4913-8a38-f288d3bffa91 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:21:22.162885 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e1d62-b7a4-4913-8a38-f288d3bffa91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:21:22.211174 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-302e1d62-b7a4-4913-8a38-f288d3bffa91 None None] MFA Rules not processed for user `d89b729776114f7ab97c7a99f5a777c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 09:21:22.212360 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-302e1d62-b7a4-4913-8a38-f288d3bffa91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:21:22.228186 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1452/2914] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 09:21:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 09:21:22.466318 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175d697-f215-4fd2-851c-5af6cae47f6f service glance] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:21:22.476413 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175d697-f215-4fd2-851c-5af6cae47f6f service glance] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:21:22.479929 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d7f41d-ce95-48c4-8c3a-d4a9d5588767 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:21:22.480508 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7f41d-ce95-48c4-8c3a-d4a9d5588767 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:21:22.480782 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7f41d-ce95-48c4-8c3a-d4a9d5588767 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:21:22.481053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7f41d-ce95-48c4-8c3a-d4a9d5588767 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:21:22.483517 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-26d7f41d-ce95-48c4-8c3a-d4a9d5588767 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:21:22.493098 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d7f41d-ce95-48c4-8c3a-d4a9d5588767 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:21:22.494530 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d7f41d-ce95-48c4-8c3a-d4a9d5588767 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:21:22.503260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1463/2915] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:21:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 09:21:22.922948 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e1d62-b7a4-4913-8a38-f288d3bffa91 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:21:22.924387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-302e1d62-b7a4-4913-8a38-f288d3bffa91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:21:22.940969 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e1d62-b7a4-4913-8a38-f288d3bffa91 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:21:22.946387 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf6110-b236-4139-b17b-5dc2a40e330d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c23a782b9b4144bace1248c3b2e754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:21:22.946970 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf6110-b236-4139-b17b-5dc2a40e330d service placement] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:21:22.947846 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf6110-b236-4139-b17b-5dc2a40e330d service placement] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:21:22.948158 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf6110-b236-4139-b17b-5dc2a40e330d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:21:22.950009 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5edf6110-b236-4139-b17b-5dc2a40e330d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:21:22.962847 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf6110-b236-4139-b17b-5dc2a40e330d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:21:22.964495 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf6110-b236-4139-b17b-5dc2a40e330d service placement] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:21:22.977069 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1453/2916] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:21:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:21:25.784752 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d7f41d-ce95-48c4-8c3a-d4a9d5588767 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:21:25.793645 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d7f41d-ce95-48c4-8c3a-d4a9d5588767 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:21:25.797893 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec07b1-5ec2-4eca-9106-89996dab7463 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:21:25.798486 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec07b1-5ec2-4eca-9106-89996dab7463 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:21:25.798762 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec07b1-5ec2-4eca-9106-89996dab7463 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:21:25.799031 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec07b1-5ec2-4eca-9106-89996dab7463 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:21:25.800720 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-baec07b1-5ec2-4eca-9106-89996dab7463 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:21:25.810459 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baec07b1-5ec2-4eca-9106-89996dab7463 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:21:25.811983 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baec07b1-5ec2-4eca-9106-89996dab7463 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:21:25.820037 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1464/2917] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:21:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:21:26.622373 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf6110-b236-4139-b17b-5dc2a40e330d service placement] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:21:26.623850 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-5edf6110-b236-4139-b17b-5dc2a40e330d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:21:26.641324 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf6110-b236-4139-b17b-5dc2a40e330d service placement] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:21:26.650560 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd94ad-c25f-43e3-be0c-e59c2780c5e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:21:26.650560 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd94ad-c25f-43e3-be0c-e59c2780c5e1 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:21:26.650560 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd94ad-c25f-43e3-be0c-e59c2780c5e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:21:26.650560 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd94ad-c25f-43e3-be0c-e59c2780c5e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:21:26.650935 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-7cfd94ad-c25f-43e3-be0c-e59c2780c5e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:21:26.663030 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfd94ad-c25f-43e3-be0c-e59c2780c5e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:21:26.664639 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfd94ad-c25f-43e3-be0c-e59c2780c5e1 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:21:26.676341 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1454/2918] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:21:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:21:28.666204 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec07b1-5ec2-4eca-9106-89996dab7463 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:21:28.679906 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec07b1-5ec2-4eca-9106-89996dab7463 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:21:28.685313 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589320e3-2d0d-44a5-beb4-34a1a79e80b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:21:28.685908 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589320e3-2d0d-44a5-beb4-34a1a79e80b9 service nova] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:21:28.686188 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589320e3-2d0d-44a5-beb4-34a1a79e80b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:21:28.686188 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589320e3-2d0d-44a5-beb4-34a1a79e80b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:21:28.694607 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589320e3-2d0d-44a5-beb4-34a1a79e80b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:21:28.696169 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589320e3-2d0d-44a5-beb4-34a1a79e80b9 service nova] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:21:28.705083 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1465/2919] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:21:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:21:33.513770 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd94ad-c25f-43e3-be0c-e59c2780c5e1 service nova] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:21:33.515773 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-7cfd94ad-c25f-43e3-be0c-e59c2780c5e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:21:33.530341 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd94ad-c25f-43e3-be0c-e59c2780c5e1 service nova] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:21:33.536098 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fac519-73e5-4350-9da9-67da9bd5a208 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3317f751ec74bde95ec262394001292', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:21:33.536791 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fac519-73e5-4350-9da9-67da9bd5a208 service nova] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:21:33.537133 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fac519-73e5-4350-9da9-67da9bd5a208 service nova] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:21:33.537456 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fac519-73e5-4350-9da9-67da9bd5a208 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:21:33.539184 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-a4fac519-73e5-4350-9da9-67da9bd5a208 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:21:33.552270 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fac519-73e5-4350-9da9-67da9bd5a208 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:21:33.554151 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fac519-73e5-4350-9da9-67da9bd5a208 service nova] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:21:33.566413 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1455/2920] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:21:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:21:38.700954 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=51 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:21:43.561358 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=64 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:22:31.842149 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589320e3-2d0d-44a5-beb4-34a1a79e80b9 service nova] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:22:31.852157 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589320e3-2d0d-44a5-beb4-34a1a79e80b9 service nova] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:22:31.855679 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d491a3-4afa-4c0f-8f77-1c0719446221 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:22:31.856260 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d491a3-4afa-4c0f-8f77-1c0719446221 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:22:31.856540 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d491a3-4afa-4c0f-8f77-1c0719446221 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:22:31.856808 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d491a3-4afa-4c0f-8f77-1c0719446221 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:22:31.858658 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f8d491a3-4afa-4c0f-8f77-1c0719446221 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:22:31.923560 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d491a3-4afa-4c0f-8f77-1c0719446221 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:22:31.924974 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d491a3-4afa-4c0f-8f77-1c0719446221 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:22:31.934442 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1466/2921] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:22:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:22:32.465240 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fac519-73e5-4350-9da9-67da9bd5a208 service nova] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:22:32.475418 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fac519-73e5-4350-9da9-67da9bd5a208 service nova] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:22:32.482801 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb18e0e-10ca-4119-9af5-0bdd370832ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c23a782b9b4144bace1248c3b2e754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:22:32.483408 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb18e0e-10ca-4119-9af5-0bdd370832ab service placement] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:22:32.483782 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb18e0e-10ca-4119-9af5-0bdd370832ab service placement] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:22:32.484224 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb18e0e-10ca-4119-9af5-0bdd370832ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:22:32.486063 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-fbb18e0e-10ca-4119-9af5-0bdd370832ab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:22:32.498724 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb18e0e-10ca-4119-9af5-0bdd370832ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:22:32.501180 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb18e0e-10ca-4119-9af5-0bdd370832ab service placement] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:22:32.511931 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1456/2922] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:22:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:22:32.588816 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d491a3-4afa-4c0f-8f77-1c0719446221 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:22:32.590904 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-f8d491a3-4afa-4c0f-8f77-1c0719446221 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:22:32.611077 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d491a3-4afa-4c0f-8f77-1c0719446221 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:22:32.614547 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45591c70-1a17-4495-8e39-25e52420b7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:22:32.615258 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45591c70-1a17-4495-8e39-25e52420b7d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:22:32.615540 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45591c70-1a17-4495-8e39-25e52420b7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:22:32.615835 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45591c70-1a17-4495-8e39-25e52420b7d5 admin admin] PATH_INFO: `/v3/users/d89b729776114f7ab97c7a99f5a777c2` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:22:32.616992 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45591c70-1a17-4495-8e39-25e52420b7d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d89b729776114f7ab97c7a99f5a777c2)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:22:32.619006 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45591c70-1a17-4495-8e39-25e52420b7d5 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:22:32.816020 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-45591c70-1a17-4495-8e39-25e52420b7d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd89b729776114f7ab97c7a99f5a777c2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:22:32.816621 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-45591c70-1a17-4495-8e39-25e52420b7d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd89b729776114f7ab97c7a99f5a777c2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:22:32.817337 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-45591c70-1a17-4495-8e39-25e52420b7d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd89b729776114f7ab97c7a99f5a777c2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:22:32.824333 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-45591c70-1a17-4495-8e39-25e52420b7d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd89b729776114f7ab97c7a99f5a777c2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:22:32.898646 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-45591c70-1a17-4495-8e39-25e52420b7d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd89b729776114f7ab97c7a99f5a777c2'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:22:32.916667 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1467/2923] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 09:22:32 2022] DELETE /identity/v3/users/d89b729776114f7ab97c7a99f5a777c2 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 09:22:33.182391 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb18e0e-10ca-4119-9af5-0bdd370832ab service placement] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:22:33.194896 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb18e0e-10ca-4119-9af5-0bdd370832ab service placement] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:22:33.255587 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5454a55-6e7e-4f87-a2f7-3cd5bcc9a8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:22:33.256220 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5454a55-6e7e-4f87-a2f7-3cd5bcc9a8c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:22:33.256533 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5454a55-6e7e-4f87-a2f7-3cd5bcc9a8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:22:33.256802 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5454a55-6e7e-4f87-a2f7-3cd5bcc9a8c0 admin admin] PATH_INFO: `/v3/projects/b0ea80c4a39f47e1a9f3d050c46fe2bb` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:22:33.258066 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5454a55-6e7e-4f87-a2f7-3cd5bcc9a8c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0ea80c4a39f47e1a9f3d050c46fe2bb)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:22:33.259832 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5454a55-6e7e-4f87-a2f7-3cd5bcc9a8c0 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:22:33.395983 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f5454a55-6e7e-4f87-a2f7-3cd5bcc9a8c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0ea80c4a39f47e1a9f3d050c46fe2bb', 'request_id': 'req-f5454a55-6e7e-4f87-a2f7-3cd5bcc9a8c0'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:22:33.402313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f5454a55-6e7e-4f87-a2f7-3cd5bcc9a8c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0ea80c4a39f47e1a9f3d050c46fe2bb', 'request_id': 'req-f5454a55-6e7e-4f87-a2f7-3cd5bcc9a8c0'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:22:33.408543 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f5454a55-6e7e-4f87-a2f7-3cd5bcc9a8c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0ea80c4a39f47e1a9f3d050c46fe2bb', 'request_id': 'req-f5454a55-6e7e-4f87-a2f7-3cd5bcc9a8c0'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:22:33.491703 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f5454a55-6e7e-4f87-a2f7-3cd5bcc9a8c0 admin admin] The token cache is being invalidate because project b0ea80c4a39f47e1a9f3d050c46fe2bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 09:22:33.492300 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f5454a55-6e7e-4f87-a2f7-3cd5bcc9a8c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:22:33.492546 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-f5454a55-6e7e-4f87-a2f7-3cd5bcc9a8c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:22:33.494153 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1457/2924] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 09:22:33 2022] DELETE /identity/v3/projects/b0ea80c4a39f47e1a9f3d050c46fe2bb => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 09:22:33.499123 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffdb66-e837-457d-a10d-563a5b1d576f None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:22:33.502278 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffdb66-e837-457d-a10d-563a5b1d576f None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:22:33.502278 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffdb66-e837-457d-a10d-563a5b1d576f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:22:33.522216 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-e0ffdb66-e837-457d-a10d-563a5b1d576f None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 09:22:33.523408 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-e0ffdb66-e837-457d-a10d-563a5b1d576f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:22:33.581819 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1468/2925] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 09:22:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 09:22:33.590792 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201d2f5-8010-4891-9d7a-a278356d4dbb None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:22:33.591100 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201d2f5-8010-4891-9d7a-a278356d4dbb None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:22:33.591395 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201d2f5-8010-4891-9d7a-a278356d4dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:22:33.614382 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-e201d2f5-8010-4891-9d7a-a278356d4dbb None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 09:22:33.622432 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-e201d2f5-8010-4891-9d7a-a278356d4dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:22:33.629103 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1458/2926] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 09:22:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 09:22:33.633450 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ffdb66-e837-457d-a10d-563a5b1d576f None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:22:33.644371 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ffdb66-e837-457d-a10d-563a5b1d576f None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:22:33.648962 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca316c-5305-4da4-954b-ff6e9dbe3692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:22:33.649613 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca316c-5305-4da4-954b-ff6e9dbe3692 admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:22:33.649887 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca316c-5305-4da4-954b-ff6e9dbe3692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:22:33.650153 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca316c-5305-4da4-954b-ff6e9dbe3692 admin admin] PATH_INFO: `/v3/domains` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:22:33.650791 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca316c-5305-4da4-954b-ff6e9dbe3692 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80261) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 09:22:33.651200 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca316c-5305-4da4-954b-ff6e9dbe3692 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:22:33.652784 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca316c-5305-4da4-954b-ff6e9dbe3692 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:22:33.654566 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:22:33.665599 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1469/2927] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 09:22:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 09:22:33.669926 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e201d2f5-8010-4891-9d7a-a278356d4dbb None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:22:33.680146 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e201d2f5-8010-4891-9d7a-a278356d4dbb None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:22:33.684870 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447fbba-7343-40a6-9924-3bec4746750d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:22:33.685520 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447fbba-7343-40a6-9924-3bec4746750d admin admin] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:22:33.685840 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447fbba-7343-40a6-9924-3bec4746750d admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:22:33.686132 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447fbba-7343-40a6-9924-3bec4746750d admin admin] PATH_INFO: `/v3/projects` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:22:33.686821 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b447fbba-7343-40a6-9924-3bec4746750d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:22:33.689219 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b447fbba-7343-40a6-9924-3bec4746750d admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:22:33.845820 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1459/2928] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 09:22:33 2022] POST /identity/v3/projects => generated 389 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 09:22:33.849979 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca316c-5305-4da4-954b-ff6e9dbe3692 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:22:33.859891 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca316c-5305-4da4-954b-ff6e9dbe3692 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:22:33.916247 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fdb94c-3713-4b50-8734-b577a2ce020e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:22:33.916923 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fdb94c-3713-4b50-8734-b577a2ce020e admin admin] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:22:33.917125 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fdb94c-3713-4b50-8734-b577a2ce020e admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:22:33.917424 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fdb94c-3713-4b50-8734-b577a2ce020e admin admin] PATH_INFO: `/v3/users` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:22:33.918111 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fdb94c-3713-4b50-8734-b577a2ce020e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:22:33.920024 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fdb94c-3713-4b50-8734-b577a2ce020e admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:22:34.021543 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1470/2929] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 09:22:33 2022] POST /identity/v3/users => generated 295 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 09:22:34.025637 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447fbba-7343-40a6-9924-3bec4746750d admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:22:34.036019 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447fbba-7343-40a6-9924-3bec4746750d admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:22:34.040601 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c27d3-afcc-4479-975d-ac10bc580652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:22:34.041233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c27d3-afcc-4479-975d-ac10bc580652 admin admin] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:22:34.041514 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c27d3-afcc-4479-975d-ac10bc580652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:22:34.041811 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c27d3-afcc-4479-975d-ac10bc580652 admin admin] PATH_INFO: `/v3/roles` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:22:34.042464 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c27d3-afcc-4479-975d-ac10bc580652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:22:34.044040 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c27d3-afcc-4479-975d-ac10bc580652 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:22:34.054218 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1460/2930] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 09:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 09:22:34.058020 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fdb94c-3713-4b50-8734-b577a2ce020e admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:22:34.069099 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fdb94c-3713-4b50-8734-b577a2ce020e admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:22:34.073889 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12998b0b-50ac-4fa3-b305-f02eea6a149c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:22:34.074494 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12998b0b-50ac-4fa3-b305-f02eea6a149c admin admin] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:22:34.074779 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12998b0b-50ac-4fa3-b305-f02eea6a149c admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:22:34.075053 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12998b0b-50ac-4fa3-b305-f02eea6a149c admin admin] PATH_INFO: `/v3/roles` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:22:34.075698 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12998b0b-50ac-4fa3-b305-f02eea6a149c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:22:34.077341 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12998b0b-50ac-4fa3-b305-f02eea6a149c admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:22:34.090435 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1471/2931] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 09:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 09:22:34.095004 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c27d3-afcc-4479-975d-ac10bc580652 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:22:34.104993 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c27d3-afcc-4479-975d-ac10bc580652 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:22:34.110461 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ab9bb-fbec-4b9c-aa57-b0e7551b397e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:22:34.111108 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ab9bb-fbec-4b9c-aa57-b0e7551b397e admin admin] REQUEST_METHOD: `PUT` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:22:34.111389 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ab9bb-fbec-4b9c-aa57-b0e7551b397e admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:22:34.111663 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ab9bb-fbec-4b9c-aa57-b0e7551b397e admin admin] PATH_INFO: `/v3/projects/4ffdde9d07774290b92b5edc92fb9977/users/368f2b1fe6024010af3ed0faf25a64e3/roles/b0ba6e2bf793402abe2e680c2a7dcb53` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:22:34.126673 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ab9bb-fbec-4b9c-aa57-b0e7551b397e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ffdde9d07774290b92b5edc92fb9977, user_id=368f2b1fe6024010af3ed0faf25a64e3, role_id=b0ba6e2bf793402abe2e680c2a7dcb53)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:22:34.130009 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ab9bb-fbec-4b9c-aa57-b0e7551b397e admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:22:34.209420 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1461/2932] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 09:22:34 2022] PUT /identity/v3/projects/4ffdde9d07774290b92b5edc92fb9977/users/368f2b1fe6024010af3ed0faf25a64e3/roles/b0ba6e2bf793402abe2e680c2a7dcb53 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 09:22:34.219711 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d4e1f-1d87-4473-8583-9b27a64a41f6 None None] REQUEST_METHOD: `POST` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:22:34.220003 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d4e1f-1d87-4473-8583-9b27a64a41f6 None None] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:22:34.220276 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d4e1f-1d87-4473-8583-9b27a64a41f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:22:34.246892 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.auth.core [None req-fa3d4e1f-1d87-4473-8583-9b27a64a41f6 None None] MFA Rules not processed for user `368f2b1fe6024010af3ed0faf25a64e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80261) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 09:22:34.248090 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-fa3d4e1f-1d87-4473-8583-9b27a64a41f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:22:34.313386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1472/2933] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 09:22:34 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 09:22:34.323853 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ab9bb-fbec-4b9c-aa57-b0e7551b397e admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:22:34.328334 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-550ab9bb-fbec-4b9c-aa57-b0e7551b397e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:22:34.343610 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:22:34.394554 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ab9bb-fbec-4b9c-aa57-b0e7551b397e admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:22:34.398255 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f498b76-f275-4646-afb3-9682a65f1faf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3262fa2d9c0d4d8998c9697994b48fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:22:34.398893 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f498b76-f275-4646-afb3-9682a65f1faf service cinder] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:22:34.399166 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f498b76-f275-4646-afb3-9682a65f1faf service cinder] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:22:34.399438 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f498b76-f275-4646-afb3-9682a65f1faf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:22:34.407655 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f498b76-f275-4646-afb3-9682a65f1faf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:22:34.409074 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f498b76-f275-4646-afb3-9682a65f1faf service cinder] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:22:34.418410 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1462/2934] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:22:34 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:22:44.309594 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=69,keystone:INSERT=3 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:22:44.446860 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=25 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:22:55.422093 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d4e1f-1d87-4473-8583-9b27a64a41f6 None None] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:22:55.484842 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d4e1f-1d87-4473-8583-9b27a64a41f6 None None] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:22:55.490119 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e2663-3cd0-4c3e-8899-a6aa1b659096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:22:55.490737 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e2663-3cd0-4c3e-8899-a6aa1b659096 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:22:55.491010 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e2663-3cd0-4c3e-8899-a6aa1b659096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:22:55.491279 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e2663-3cd0-4c3e-8899-a6aa1b659096 admin admin] PATH_INFO: `/v3/users/368f2b1fe6024010af3ed0faf25a64e3` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:22:55.492376 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e2663-3cd0-4c3e-8899-a6aa1b659096 admin admin] RBAC: Authorizing `identity:delete_user(user_id=368f2b1fe6024010af3ed0faf25a64e3)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:22:55.494301 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e2663-3cd0-4c3e-8899-a6aa1b659096 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:22:55.706121 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ab2e2663-3cd0-4c3e-8899-a6aa1b659096 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '368f2b1fe6024010af3ed0faf25a64e3'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:22:55.706768 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ab2e2663-3cd0-4c3e-8899-a6aa1b659096 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '368f2b1fe6024010af3ed0faf25a64e3'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:22:55.707367 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ab2e2663-3cd0-4c3e-8899-a6aa1b659096 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '368f2b1fe6024010af3ed0faf25a64e3'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:22:55.714712 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ab2e2663-3cd0-4c3e-8899-a6aa1b659096 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '368f2b1fe6024010af3ed0faf25a64e3'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:22:55.772351 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-ab2e2663-3cd0-4c3e-8899-a6aa1b659096 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '368f2b1fe6024010af3ed0faf25a64e3'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:22:55.790386 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1473/2935] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 09:22:55 2022] DELETE /identity/v3/users/368f2b1fe6024010af3ed0faf25a64e3 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 09:22:55.794764 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc161dbe-4c43-4997-a12c-564e99e8d008 None None] REQUEST_METHOD: `POST` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:22:55.795284 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc161dbe-4c43-4997-a12c-564e99e8d008 None None] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:22:55.795284 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc161dbe-4c43-4997-a12c-564e99e8d008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:22:55.814527 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.auth.core [None req-dc161dbe-4c43-4997-a12c-564e99e8d008 None None] MFA Rules not processed for user `63f02a13b768446ca706d7594e04d3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80262) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 09:22:55.815817 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-dc161dbe-4c43-4997-a12c-564e99e8d008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:22:55.873413 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1463/2936] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 09:22:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 09:22:55.883170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e2663-3cd0-4c3e-8899-a6aa1b659096 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:22:55.884567 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-ab2e2663-3cd0-4c3e-8899-a6aa1b659096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:22:55.943796 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e2663-3cd0-4c3e-8899-a6aa1b659096 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:22:55.947271 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ba565-3016-442a-a8e7-bdb09164c5fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:22:55.947865 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ba565-3016-442a-a8e7-bdb09164c5fe service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:22:55.948139 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ba565-3016-442a-a8e7-bdb09164c5fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:22:55.948407 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ba565-3016-442a-a8e7-bdb09164c5fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:22:55.957643 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ba565-3016-442a-a8e7-bdb09164c5fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:22:55.959062 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ba565-3016-442a-a8e7-bdb09164c5fe service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:22:55.968183 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1474/2937] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:22:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:22:56.394717 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc161dbe-4c43-4997-a12c-564e99e8d008 None None] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:22:56.396227 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-dc161dbe-4c43-4997-a12c-564e99e8d008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:22:56.408233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc161dbe-4c43-4997-a12c-564e99e8d008 None None] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:22:56.416409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa36b7-fd8c-4e51-b3a2-dd47ddd0138d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:22:56.417095 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa36b7-fd8c-4e51-b3a2-dd47ddd0138d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:22:56.418947 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa36b7-fd8c-4e51-b3a2-dd47ddd0138d admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:22:56.419233 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa36b7-fd8c-4e51-b3a2-dd47ddd0138d admin admin] PATH_INFO: `/v3/projects/4ffdde9d07774290b92b5edc92fb9977` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:22:56.420836 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baa36b7-fd8c-4e51-b3a2-dd47ddd0138d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ffdde9d07774290b92b5edc92fb9977)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:22:56.422682 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baa36b7-fd8c-4e51-b3a2-dd47ddd0138d admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:22:56.538902 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-8baa36b7-fd8c-4e51-b3a2-dd47ddd0138d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ffdde9d07774290b92b5edc92fb9977', 'request_id': 'req-8baa36b7-fd8c-4e51-b3a2-dd47ddd0138d'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:22:56.544930 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-8baa36b7-fd8c-4e51-b3a2-dd47ddd0138d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ffdde9d07774290b92b5edc92fb9977', 'request_id': 'req-8baa36b7-fd8c-4e51-b3a2-dd47ddd0138d'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:22:56.551335 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-8baa36b7-fd8c-4e51-b3a2-dd47ddd0138d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ffdde9d07774290b92b5edc92fb9977', 'request_id': 'req-8baa36b7-fd8c-4e51-b3a2-dd47ddd0138d'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:22:56.621389 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-8baa36b7-fd8c-4e51-b3a2-dd47ddd0138d admin admin] The token cache is being invalidate because project 4ffdde9d07774290b92b5edc92fb9977 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 09:22:56.626162 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-8baa36b7-fd8c-4e51-b3a2-dd47ddd0138d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:22:56.626162 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-8baa36b7-fd8c-4e51-b3a2-dd47ddd0138d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:22:56.626162 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1464/2938] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 09:22:56 2022] DELETE /identity/v3/projects/4ffdde9d07774290b92b5edc92fb9977 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 09:23:05.964056 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:23:06.618179 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:24:28.713841 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ba565-3016-442a-a8e7-bdb09164c5fe service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:24:28.733312 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ba565-3016-442a-a8e7-bdb09164c5fe service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:24:28.790556 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd581ee5-b975-4a12-9c4d-5a2ead4cf955 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3b8b43026c40209bfc42acd2830610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:24:28.791322 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd581ee5-b975-4a12-9c4d-5a2ead4cf955 service neutron] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:24:28.791722 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd581ee5-b975-4a12-9c4d-5a2ead4cf955 service neutron] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:24:28.792002 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd581ee5-b975-4a12-9c4d-5a2ead4cf955 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:24:28.796803 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-dd581ee5-b975-4a12-9c4d-5a2ead4cf955 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:24:28.882486 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd581ee5-b975-4a12-9c4d-5a2ead4cf955 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:24:28.884174 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd581ee5-b975-4a12-9c4d-5a2ead4cf955 service neutron] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:24:28.902669 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1475/2939] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:24:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:24:35.870409 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa36b7-fd8c-4e51-b3a2-dd47ddd0138d admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:24:35.871877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-8baa36b7-fd8c-4e51-b3a2-dd47ddd0138d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:24:35.958187 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa36b7-fd8c-4e51-b3a2-dd47ddd0138d admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:24:35.963323 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95fc5aa-ae40-433d-8275-264671a41968 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c23a782b9b4144bace1248c3b2e754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:24:35.963936 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fc5aa-ae40-433d-8275-264671a41968 service placement] REQUEST_METHOD: `GET` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:24:35.964215 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fc5aa-ae40-433d-8275-264671a41968 service placement] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:24:35.964484 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fc5aa-ae40-433d-8275-264671a41968 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:24:35.966268 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-d95fc5aa-ae40-433d-8275-264671a41968 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:24:35.978208 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95fc5aa-ae40-433d-8275-264671a41968 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:24:35.979815 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95fc5aa-ae40-433d-8275-264671a41968 service placement] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:24:35.998808 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1465/2940] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:24:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:24:35.999087 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd581ee5-b975-4a12-9c4d-5a2ead4cf955 service neutron] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:24:35.999087 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-dd581ee5-b975-4a12-9c4d-5a2ead4cf955 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:24:36.009549 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd581ee5-b975-4a12-9c4d-5a2ead4cf955 service neutron] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:24:36.014618 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d620954-df36-4fd7-beaf-7734775ebc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:24:36.015327 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d620954-df36-4fd7-beaf-7734775ebc22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:24:36.015605 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d620954-df36-4fd7-beaf-7734775ebc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:24:36.015879 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d620954-df36-4fd7-beaf-7734775ebc22 admin admin] PATH_INFO: `/v3/users/cd870619247a47ac810fea1aa636ac65` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:24:36.017122 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d620954-df36-4fd7-beaf-7734775ebc22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd870619247a47ac810fea1aa636ac65)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:24:36.019288 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d620954-df36-4fd7-beaf-7734775ebc22 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:24:36.149916 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1d620954-df36-4fd7-beaf-7734775ebc22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd870619247a47ac810fea1aa636ac65'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:24:36.150566 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1d620954-df36-4fd7-beaf-7734775ebc22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd870619247a47ac810fea1aa636ac65'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:24:36.151177 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1d620954-df36-4fd7-beaf-7734775ebc22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd870619247a47ac810fea1aa636ac65'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:24:36.158671 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1d620954-df36-4fd7-beaf-7734775ebc22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd870619247a47ac810fea1aa636ac65'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:24:36.198914 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-1d620954-df36-4fd7-beaf-7734775ebc22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd870619247a47ac810fea1aa636ac65'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:24:36.204170 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:24:36.217099 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1476/2941] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 09:24:35 2022] DELETE /identity/v3/users/cd870619247a47ac810fea1aa636ac65 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 09:24:36.450320 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95fc5aa-ae40-433d-8275-264671a41968 service placement] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:24:36.462755 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95fc5aa-ae40-433d-8275-264671a41968 service placement] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:24:36.527204 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f85af5-aba3-436a-9bf4-98d7d1696ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:24:36.528030 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f85af5-aba3-436a-9bf4-98d7d1696ea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:24:36.528313 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f85af5-aba3-436a-9bf4-98d7d1696ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:24:36.528581 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f85af5-aba3-436a-9bf4-98d7d1696ea9 admin admin] PATH_INFO: `/v3/projects/e87616891fa34cd9aca388a5d101f3f9` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:24:36.529722 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f85af5-aba3-436a-9bf4-98d7d1696ea9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e87616891fa34cd9aca388a5d101f3f9)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:24:36.532331 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f85af5-aba3-436a-9bf4-98d7d1696ea9 admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:24:36.626221 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-62f85af5-aba3-436a-9bf4-98d7d1696ea9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e87616891fa34cd9aca388a5d101f3f9', 'request_id': 'req-62f85af5-aba3-436a-9bf4-98d7d1696ea9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:24:36.632522 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-62f85af5-aba3-436a-9bf4-98d7d1696ea9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e87616891fa34cd9aca388a5d101f3f9', 'request_id': 'req-62f85af5-aba3-436a-9bf4-98d7d1696ea9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:24:36.637984 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-62f85af5-aba3-436a-9bf4-98d7d1696ea9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e87616891fa34cd9aca388a5d101f3f9', 'request_id': 'req-62f85af5-aba3-436a-9bf4-98d7d1696ea9'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:24:36.712235 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-62f85af5-aba3-436a-9bf4-98d7d1696ea9 admin admin] The token cache is being invalidate because project e87616891fa34cd9aca388a5d101f3f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 09:24:36.712525 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-62f85af5-aba3-436a-9bf4-98d7d1696ea9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:24:36.713263 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-62f85af5-aba3-436a-9bf4-98d7d1696ea9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:24:36.714887 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1466/2942] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 09:24:36 2022] DELETE /identity/v3/projects/e87616891fa34cd9aca388a5d101f3f9 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 09:24:36.719539 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d620954-df36-4fd7-beaf-7734775ebc22 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:24:36.720955 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1d620954-df36-4fd7-beaf-7734775ebc22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:24:36.735289 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d620954-df36-4fd7-beaf-7734775ebc22 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:24:36.793527 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2dde15-ccb7-4b28-8cae-0bb109ba6616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:24:36.794190 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2dde15-ccb7-4b28-8cae-0bb109ba6616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:24:36.794470 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2dde15-ccb7-4b28-8cae-0bb109ba6616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:24:36.794742 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2dde15-ccb7-4b28-8cae-0bb109ba6616 admin admin] PATH_INFO: `/v3/users/fd6aaf65a27c4e95ad96508ad1288901` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:24:36.808061 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2dde15-ccb7-4b28-8cae-0bb109ba6616 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd6aaf65a27c4e95ad96508ad1288901)` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:24:36.810032 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2dde15-ccb7-4b28-8cae-0bb109ba6616 admin admin] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:24:36.960743 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bd2dde15-ccb7-4b28-8cae-0bb109ba6616 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd6aaf65a27c4e95ad96508ad1288901'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:24:36.961344 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bd2dde15-ccb7-4b28-8cae-0bb109ba6616 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd6aaf65a27c4e95ad96508ad1288901'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:24:36.961944 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bd2dde15-ccb7-4b28-8cae-0bb109ba6616 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd6aaf65a27c4e95ad96508ad1288901'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:24:36.969523 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bd2dde15-ccb7-4b28-8cae-0bb109ba6616 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd6aaf65a27c4e95ad96508ad1288901'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:24:37.021216 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.notifications [None req-bd2dde15-ccb7-4b28-8cae-0bb109ba6616 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd6aaf65a27c4e95ad96508ad1288901'} {{(pid=80261) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:24:37.038319 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1477/2943] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 09:24:36 2022] DELETE /identity/v3/users/fd6aaf65a27c4e95ad96508ad1288901 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 09:24:37.278661 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f85af5-aba3-436a-9bf4-98d7d1696ea9 admin admin] Authenticating user token {{(pid=80262) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:24:37.280696 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.fernet_utils [None req-62f85af5-aba3-436a-9bf4-98d7d1696ea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80262) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:24:37.297252 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f85af5-aba3-436a-9bf4-98d7d1696ea9 admin admin] Validating token access rules against request {{(pid=80262) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:24:37.314996 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:24:37.354658 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0cb1fc-eff2-4c9b-ae05-6b2f5ca5312f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63f02a13b768446ca706d7594e04d3fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '785791bda4044663bfe6cd25439a8358', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80262) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:24:37.355288 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cb1fc-eff2-4c9b-ae05-6b2f5ca5312f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:24:37.355565 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cb1fc-eff2-4c9b-ae05-6b2f5ca5312f admin admin] SCRIPT_NAME: `/identity` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:24:37.355837 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cb1fc-eff2-4c9b-ae05-6b2f5ca5312f admin admin] PATH_INFO: `/v3/projects/02641b8d4ca7466bb4fb396a9fdf5e11` {{(pid=80262) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:24:37.367239 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0cb1fc-eff2-4c9b-ae05-6b2f5ca5312f admin admin] RBAC: Authorizing `identity:delete_project(project_id=02641b8d4ca7466bb4fb396a9fdf5e11)` {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:24:37.369242 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0cb1fc-eff2-4c9b-ae05-6b2f5ca5312f admin admin] RBAC: Authorization granted {{(pid=80262) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:24:37.503229 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-fc0cb1fc-eff2-4c9b-ae05-6b2f5ca5312f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02641b8d4ca7466bb4fb396a9fdf5e11', 'request_id': 'req-fc0cb1fc-eff2-4c9b-ae05-6b2f5ca5312f'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:24:37.510135 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-fc0cb1fc-eff2-4c9b-ae05-6b2f5ca5312f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02641b8d4ca7466bb4fb396a9fdf5e11', 'request_id': 'req-fc0cb1fc-eff2-4c9b-ae05-6b2f5ca5312f'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:24:37.515808 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-fc0cb1fc-eff2-4c9b-ae05-6b2f5ca5312f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02641b8d4ca7466bb4fb396a9fdf5e11', 'request_id': 'req-fc0cb1fc-eff2-4c9b-ae05-6b2f5ca5312f'} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:24:37.586662 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-fc0cb1fc-eff2-4c9b-ae05-6b2f5ca5312f admin admin] The token cache is being invalidate because project 02641b8d4ca7466bb4fb396a9fdf5e11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80262) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 09:24:37.586877 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-fc0cb1fc-eff2-4c9b-ae05-6b2f5ca5312f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:24:37.587493 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG keystone.notifications [None req-fc0cb1fc-eff2-4c9b-ae05-6b2f5ca5312f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80262) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 09:24:37.589001 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: [pid: 80262|app: 0|req: 1467/2944] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 09:24:37 2022] DELETE /identity/v3/projects/02641b8d4ca7466bb4fb396a9fdf5e11 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 09:24:47.035208 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:DELETE=12,keystone:SELECT=63,keystone:INSERT=1 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:24:47.583247 dv-cinder-836973-17-iscsi devstack@keystone.service[80262]: DEBUG dbcounter [-] [80262] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80262) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 09:26:20.131464 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2dde15-ccb7-4b28-8cae-0bb109ba6616 admin admin] Authenticating user token {{(pid=80261) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 09:26:20.132903 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-bd2dde15-ccb7-4b28-8cae-0bb109ba6616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:26:20.200508 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2dde15-ccb7-4b28-8cae-0bb109ba6616 admin admin] Validating token access rules against request {{(pid=80261) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 09:26:20.258344 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac64276-8c50-4e0b-b573-17d237d1b224 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98c23a782b9b4144bace1248c3b2e754', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71e5e82c9f454cdd91dfeac5a57968c4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80261) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 09:26:20.258943 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac64276-8c50-4e0b-b573-17d237d1b224 service placement] REQUEST_METHOD: `GET` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 09:26:20.259217 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac64276-8c50-4e0b-b573-17d237d1b224 service placement] SCRIPT_NAME: `/identity` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 09:26:20.259485 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac64276-8c50-4e0b-b573-17d237d1b224 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80261) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 09:26:20.261173 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.fernet_utils [None req-1ac64276-8c50-4e0b-b573-17d237d1b224 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80261) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 09:26:20.272265 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac64276-8c50-4e0b-b573-17d237d1b224 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 09:26:20.273863 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac64276-8c50-4e0b-b573-17d237d1b224 service placement] RBAC: Authorization granted {{(pid=80261) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 09:26:20.283877 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: [pid: 80261|app: 0|req: 1478/2945] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 09:26:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 09:26:30.279286 dv-cinder-836973-17-iscsi devstack@keystone.service[80261]: DEBUG dbcounter [-] [80261] Writing DB stats keystone:SELECT=58 {{(pid=80261) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}